I need help architecting a news indexing system. I have clear system requirements but do not have the technical background to determine the best technologies to use. . The system I want developed would have the following functionality…
News Aggregator System
System should query major search engine’s news articles based on a predefined keyword list. The system would locally store news data gathered each day. The system would also index news feeds from a predefined list of websites w/ rss feeds. Once the data is stored in a local database a user could review and flag for deletion any unwanted articles. The remaining articles would be used to feed content to Wordpress.
Given these system requirements – how would you go about designing this system? Would you use open source software such as MagPie? I plan to post this job on a freelance site but would like to have a clear technology direction first. I hope I’m posting this in the correct forum.
Help Designing New System
Moderator: General Moderators
Re: Help Designing New System
There's no reason why php couldn't be used to do this job. Depending on what you want to do with the interface, you might also consider AJAX and jQuery (both javascript based) to help you out. I'm just learning about AJAX and jQuery myself and I'm quite impressed.
Since everything I just mentioned above is free AND has a large community of support, I see no reason why you wouldn't at least do it with PHP and mySQL.
Since everything I just mentioned above is free AND has a large community of support, I see no reason why you wouldn't at least do it with PHP and mySQL.
Re: Help Designing New System
I would use something low level like Zend_Rss. I guess just save every news entry along with he time you retrieved it. Later on when you query that data I guess you can decide how it should be ordered, and how you will "thin" out duplicate stories and such.