Scrape Ajax Search Suggest

With both Yahoo & Google search, when you start typing a search query into the search box, they return search suggestions with Ajax. See the attached screenshots. I want to scrape a certain number of these.

I want to scrape every variation of search suggest for 1 to 3 letters. So the scrape would begin with searches like this

a
aa
ab
ac
ad
……
aaa
aab
aac
aad
……
b
ba
bb
bc
bd
……
baa
bab
bac

So it’s something like 27 to the third power (26 letters and no letter as an option) entries. So Something like 20,000 entries on both Google & Yahoo.

This data set needs to have all exact duplicate results filtered out.

The returned data results would have the 4 columns
– date
– search engine (google/yahoo)
– search term used to pull search suggest (aaa, abc, hfj, ab, ect)
– search suggest keyphrase

I would liek to have the data returned in

Leave a Reply

Your email address will not be published. Required fields are marked *