I have a list of 100,000 Google Search Terms (e.g. one search term would be “Babysitter San Francisco California”), and I need you to return the all the meta-data for the first 50 results on Google’s SERP for each of these terms. That meta-data would include:
-Title
-URL Linked To
-Position on Page (1st, 2nd, 3rd, etc)
-Description – optional
Returning meta-data for ads, google maps, or anything else that’s not a search result would be great, but not required. I’d prefer to get the results in a text file with ~5 million rows (100,000 * 50).
Obviously, the trickiest part of this assignment will be doing this in a way where google will let you query them 100k times!
Alternatively, if you have experience doing this on Bing or Yahoo I’d happily accept results from there.
If this is done well, I have another 1 million search terms that I’d consider running as well, and I might go for more than just the top 50 results.
