Page 1 of 1

do you know how can i make a spider ( like google spider)

Posted: Thu Sep 18, 2008 2:53 pm
by mgtm3
i want to make a spider that can read other websites source code and give them to me

do you know how?

Re: do you know how can i make a spider ( like google spider)

Posted: Thu Sep 18, 2008 3:56 pm
by onion2k
To be brutally honest ... if you're stuck on "how can I make a spider?" rather than the "how can I make a good spider?" then you probably shouldn't bother. Even a basic spider is a lot of quite hard work, and creating a good spider (eg one that's fast and obeys rules) is very difficult.

That said though, I have seen a few open source examples, so you could try downloading some of those and see if they'll solve your problem.