Page 1 of 1

Need info about a PHP task

Posted: Mon May 31, 2010 8:37 pm
by TCJohnson
The boss is interested in having a program that will scan 20 web sites daily for specific keywords, and then return a portion of the text around the keyword and a link to that page in an email.

Our site is written in PHP currently. I know we will need to find a competant coder to get this done, but would like to know more about what exactly I am asking them to develop. Will this need to be some kind of bot program? Is there third party software that does this for us, or will this need to be a custom job? Is this a program that will run off our current site or on the desktop? And finally, how big is this task?

Thank you for any information you can offer.
TC

Re: Need info about a PHP task

Posted: Mon May 31, 2010 9:06 pm
by phdatabase
All very good questions.

One definitive question I would ask you is the source of the 20 websites a day. If you are going to give the script a list then it's a bot, if you want the script to find the sites, then it's more of a spider. Definitions aside, what you are asking to do is not difficult. Basically you load a web page, strip the links and check for the key words. If you create this as a recursive function you can easily go through a complete website. As you finish a page you can send an email with the required information, stick it in an array or database (best). You then have a whole list of links to continue on with. Trust me, you will never lack for links this way. There are plenty of third party packages out there and they are just general enough to not do exactly what you want to do. Check them out but I'll bet a custom job would not be that expensive and will more closely fit your needs.

It could run either place. I have one running on my home server right now that has been in operation for better than a month now. It's not a big project at all but you need someone who is experienced with cURL and knows how to keep your script out of trouble. 3-5 days of scripting (depending on exactly what you want) and then testing.

Amazingly enough, I specialize in agents and would certainly love to discuss this project with you if you decide on a custom solution.

Re: Need info about a PHP task

Posted: Mon May 31, 2010 9:29 pm
by TCJohnson
Currently it is a set list of 20 web sites that we want to monitor, with a predefined list of keywords, so I assume it is a bot. And yes, we are looking for a quote as well, as we currently are not sure how much this will cost.

Thanks so much for taking the time to respond.

TC

Re: Need info about a PHP task

Posted: Mon May 31, 2010 9:37 pm
by phdatabase
May I PM you to discuss for the purpose of submitting a quote?

Re: Need info about a PHP task

Posted: Mon May 31, 2010 10:58 pm
by TCJohnson
Yes, please.

Re: Need info about a PHP task

Posted: Mon May 31, 2010 11:24 pm
by Benjamin
:arrow: Moved to Job Hunt

Re: Need info about a PHP task

Posted: Wed Jun 02, 2010 2:04 pm
by phdatabase
Please check your PM for message