I’m looking to build an URL scraper with the option of rotating http/https proxy IP addresses built in and the option to just use one proxy IP as well.
This scraper must fetch random URL’s from a top million website excel spreadsheet and let me choose in lots of ie. 5,10,20,50, etc… on how many random URLs I want back. Also, want it to format If you can get one random internal page of the random URL scraped that would be great too. I don’t need all the internal pages scraped from the URL, …
