Government Bid Sites
US States, Counties, City/Towns typically have website pages where they put up current projects so that companies can bid to provide services. For example, the City of San Antonio in Texas has their information at the following website: http://www.sanantonio.gov/pgs/procure/bid_oppt.asp?res=1024&ver=true
I would like to have a script that gets all the current projects, and which will update regularly. Output should be a CSV, organized by State, County, City/Town.
There are a few stages to this project:
1) Get a complete list of States, Counties, City/Towns, and the URLs of the pages where they have their current projects, for those that do have such pages. Right now I have a list of States and City/Towns, with many URLS, which I can provide if that helps.
2) Create scripts to gather info from each of the pages when it is run, save in MySQL, and produce CSV upon request.
NOTE: This system will require updating whenever a City/Town/County/State changes its website. So it should be created in such a way as to be easily updatable. Also, if there is a change, so that the script is no longer operational for a certain website, it should produce a signal so that we can know that updating is needed.
If the price is right, and you can make this happen, will also need that maintenance work done. Probably won’t be too much maintenance work, since people don’t change websites that often.