I need an application to run in my computer, that goes to a forum section and grabs the title of each post, date, number of views and number of replies – for the entire forum section.. ALL the pages that forum section has (it doesn’t need to go inside every post.. just go through the pages and grab the titles/dates/number of views and number of replies of each post).
I need that application to let me define the interval (X hours, days, etc) when the application goes to the forum section and performs the “crawl”.
I also want a button to do it instantly when I click on it.
I need that application to let me define the forum section URL I want it to go and do the “crawl” so I can use it in multiple forums. Although I specify a single URL for the forum section , I need it do be able to go through all pages of that section, not just the URL I specify.
Please have in mind that the URL structure for different forums can be different, some can be like:
forum/forumdisplay.html?p=1
forum/forumdisplay.html?p=2
forum/forumdisplay.html?p=3 … etc
and others can be like:
forum/section/
forum/section/index1.html
forum /section/index2.html
( note that the URL structure for the first page of the forum section is usually different from the following pages ).
The application has to be able to handle all this.
That application then sends the collected titles of each post along with dates, number of views and number of replies to a webpage.
I need to be able to define the webpage I want the collected information to be sent to (the url), in the application that runs in my computer.
On that webpage I want to be able to organize that data.
I want to be able to select the interval of the collected data I want to organize (last X hours, last X days, last week, last month or all available data (the entire database).
On that webpage, I want an option where I can input a certain keyword like “forex” and it goes over all the titles (for the selected interval of time – last X days, last week, last month, all available data, etc) and only displays the titles that have “forex” in it.
This option has to be optional, not required to do the organization, I only use a keywords if I want to.
Then, I want an option to choose “most views” and it displays the titles with most views first and the ones with less views at the end.
I also want an option to choose “most replies” and it displays the titles with most replies first and the ones with less replies at the end.
These last 2 options are the main options, and they have to work when I specify a keyword (with the keyword option) as well.
I will own all the rights to the application.
I also want the source code.
If you have more questions, please let me know.
