The Script will run from the command line. It will take input and perform a series of searches on www.google.com/trends
The key functionality of this script is that it will return a file with daily results for the search terms entered. When you search on Google trends over a range of dates greater than 1 month, and download the resulting csv file, the search volume data is presented as weekly. When you search on Google trends over a range of one month, it returns data in the resulting csv file as daily volume search data. I need the daily volume data, but over longer timeframes, hence the need for this script.
The script will take as input several flags:
-q ‘search terms’; This should allow the full functionality of the search options e.g. see http://www.google.com/intl/en/trends/about.html#6
-geo region (a two letter code which can be passed to the search url); “all” by default
-s start date in format YYYY-MM; default value is 2004-1
-e end date in format YYYY-MM; default value is the most recent month
-r the output data will be with fixed scaling by default; this switch will downloads the csv files with relative scaling instead.
The script will then take the input and perform a series of searches on Google trends for each month between the start and end date. It will download the data from each of these searches as a csv file (e.g. the export “CSV with fixed scaling” button at the bottom of the search results). This file contains the daily search data that we need. Once the script has all the data (e.g. every daily search volume between start and end date), it then writes a new file containing all the data. This will be in the following format: A tab delimited file, with a header line “Date SearchTerm StdError”. It will then contain the output from the google searches, in date order. The date will be in the format YYYYMMDD, the next two fields are the search volume and StdError.
Furthermore, the script will need to be logged in to google trends, so it needs some way to handle this. Also, I’m not sure if running all these searches may cause problems with google not liking the large volume of search requests. I have heard that they might start producing captcha codes. So the script may need to overcome this. This could perhaps be achieved by just slowing down the searches. Otherwise, it could pass the captcha codes to the user. If you come across this problem, just solve it the easiest way.
Also, if a rare search term is entered, the script may come across the page “Your terms – blahblahblah – do not have enough search volume to show graphs.” In such a scenario, the data for each day of that month will be made to contain 0.
