The boss is interested in having a program that will scan 20 web sites daily for specific keywords, and then return a portion of the text around the keyword and a link to that page in an email.
Our site is written in PHP currently. I know we will need to find a competant coder to get this done, but would like to know more about what exactly I am asking them to develop. Will this need to be some kind of bot program? Is there third party software that does this for us, or will this need to be a custom job? Is this a program that will run off our current site or on the desktop? And finally, how big is this task?
Thank you for any information you can offer.
TC
Need info about a PHP task
Moderator: General Moderators
- phdatabase
- Forum Commoner
- Posts: 83
- Joined: Fri May 28, 2010 10:02 am
- Location: Fort Myers, FL
Re: Need info about a PHP task
All very good questions.
One definitive question I would ask you is the source of the 20 websites a day. If you are going to give the script a list then it's a bot, if you want the script to find the sites, then it's more of a spider. Definitions aside, what you are asking to do is not difficult. Basically you load a web page, strip the links and check for the key words. If you create this as a recursive function you can easily go through a complete website. As you finish a page you can send an email with the required information, stick it in an array or database (best). You then have a whole list of links to continue on with. Trust me, you will never lack for links this way. There are plenty of third party packages out there and they are just general enough to not do exactly what you want to do. Check them out but I'll bet a custom job would not be that expensive and will more closely fit your needs.
It could run either place. I have one running on my home server right now that has been in operation for better than a month now. It's not a big project at all but you need someone who is experienced with cURL and knows how to keep your script out of trouble. 3-5 days of scripting (depending on exactly what you want) and then testing.
Amazingly enough, I specialize in agents and would certainly love to discuss this project with you if you decide on a custom solution.
One definitive question I would ask you is the source of the 20 websites a day. If you are going to give the script a list then it's a bot, if you want the script to find the sites, then it's more of a spider. Definitions aside, what you are asking to do is not difficult. Basically you load a web page, strip the links and check for the key words. If you create this as a recursive function you can easily go through a complete website. As you finish a page you can send an email with the required information, stick it in an array or database (best). You then have a whole list of links to continue on with. Trust me, you will never lack for links this way. There are plenty of third party packages out there and they are just general enough to not do exactly what you want to do. Check them out but I'll bet a custom job would not be that expensive and will more closely fit your needs.
It could run either place. I have one running on my home server right now that has been in operation for better than a month now. It's not a big project at all but you need someone who is experienced with cURL and knows how to keep your script out of trouble. 3-5 days of scripting (depending on exactly what you want) and then testing.
Amazingly enough, I specialize in agents and would certainly love to discuss this project with you if you decide on a custom solution.
Re: Need info about a PHP task
Currently it is a set list of 20 web sites that we want to monitor, with a predefined list of keywords, so I assume it is a bot. And yes, we are looking for a quote as well, as we currently are not sure how much this will cost.
Thanks so much for taking the time to respond.
TC
Thanks so much for taking the time to respond.
TC
- phdatabase
- Forum Commoner
- Posts: 83
- Joined: Fri May 28, 2010 10:02 am
- Location: Fort Myers, FL
Re: Need info about a PHP task
May I PM you to discuss for the purpose of submitting a quote?
Re: Need info about a PHP task
Yes, please.
Re: Need info about a PHP task
- phdatabase
- Forum Commoner
- Posts: 83
- Joined: Fri May 28, 2010 10:02 am
- Location: Fort Myers, FL
Re: Need info about a PHP task
Please check your PM for message