With both Yahoo & Google search, when you start typing a search query into the search box, they return search suggestions with Ajax. See the attached screenshots. I want to scrape a certain number of these.
I want to scrape every variation of search suggest for 1 to 3 letters. So the scrape would begin with searches like this
a
aa
ab
ac
ad
……
aaa
aab
aac
aad
……
b
ba
bb
bc
bd
……
baa
bab
bac
So it’s something like 27 to the third power (26 letters and no letter as an option) entries. So Something like 20,000 entries on both Google & Yahoo.
This data set needs to have all exact duplicate results filtered out.
The returned data results would have the 4 columns
– date
– search engine (google/yahoo)
– search term used to pull search suggest (aaa, abc, hfj, ab, ect)
– search suggest keyphrase
I would liek to have the data returned in