PHP and Internet Search Crawlers/engines
Posted: Tue Apr 20, 2004 8:53 am
OK
I am looking to build a specific Internet search site and I what to use PHP. I need scripts/code/packages that will crawl or spider to a list of predetermined sites, pull keywords/metatag/titles(and such) from these site and write the parsed date to mySQL (with indexing) and then the user will search my site that is specific to my searches.
I have been search for package but I am not finding what I am looking for. I have found a package that is called Harvest that is close.
Does anyone have any ideas on the best solution. I do not want a meta search engine nor do I need a small site search engine.
Any info would be appreciated!
Thanks,
I am looking to build a specific Internet search site and I what to use PHP. I need scripts/code/packages that will crawl or spider to a list of predetermined sites, pull keywords/metatag/titles(and such) from these site and write the parsed date to mySQL (with indexing) and then the user will search my site that is specific to my searches.
I have been search for package but I am not finding what I am looking for. I have found a package that is called Harvest that is close.
Does anyone have any ideas on the best solution. I do not want a meta search engine nor do I need a small site search engine.
Any info would be appreciated!
Thanks,