Web Data Scraper
I need a scraper which will:
1. Login to a site with my username and password
2. Go to a specific ‘report’
3. Grab all the data from that report and store it in my database.
4. Allow me to view that data.
Summary of Data:
Data is in a table format and should be easily scrapable. It has column headers, which would be the same as the datatable field names.
There are 2 pages to scrape resulting in 2 separate datatables of information.
First Page Data looks like this:
http://www.sensicorp.com/staging/scraper/ScrapeData1.png
Second Page Data looks like this:
http://www.sensicorp.com/staging/scraper/ScrapeData2.png
Since this is date-sensitive information and dates can be inputted into the system – there should be the ability in the system to get older data by putting in the date range. This would be used to back-fill the database.
this should be pretty simple with Snoopy or other.
Thank you.