Advanced Scraping / Crawler
I need an advanced scraper/crawler built, it should be web-based with an admin panel and store all the scraped data to a MySQL database.
1. It should scrape a database of products from a cosmetics review site, in addition it will also scrape all the reviews for these products and the user information of the people who wrote the reviews
2. Using the product database from step 1 it will then scrape 2 additional websites for further product information such as description, image and so on.
3. It will then scrape 4 additional websites where these products are being reviewed and scrape all reviews from these websites
The scraper needs to be able to run on a regular basis to scrape any new products/reviews/product information that is added to any of the websites. As the products may be named slightly differently on each website the scraper must be able to deal with this, one way could be to allow the admin to type possible aliases for each product/and to suggest potential aliases.
Further details provided on PMB.
Please do not bid unless you have scraping experience, I am tired of people wasting my time.