Indexing Sites...
Posted: Mon Oct 06, 2003 3:55 am
I'm wanting to set up a script which indexes 100's even 1000's of different websites. This is the code I will use:
Now, if it is indexing 1000's sites at one time, it will time out right? is there anyway to avoide that?
Also, if I use a cron job to run the script, can that time out?
Thanks
Code: Select all
<?php
$query = "select * from websites";
$result = mysql_query($query);
$numrows = mysql_num_rows($result);
for($x=0;$x<$numrows;$x++) {
$row = mysql_fetch_assoc($result);
$fp = @fopen($row[url], "r" ); // Open the URL
$content = "";
if( $fp )
{
// Read in the URL
while( !feof( $fp ) )
$content .= fread( $fp, 10240 );
fclose( $fp );
}
}
?>Also, if I use a cron job to run the script, can that time out?
Thanks