Page 1 of 1

Script aborts randomly.

Posted: Tue Apr 18, 2006 1:26 pm
by Longlands
feyd | Please use

Code: Select all

,

Code: Select all

and [syntax="..."] tags where appropriate when posting code. Your post has been edited to reflect how we'd like it posted. Please read:  [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url] to learn how to do it too.[/color]


I'm writing a script that takes a list of several hundred URLs, file_get_contents the source and extracts the page's title.

Unfortunately, the script just stops running at random times (showing the page as 'Done').

By random, I mean that it will work perfectly for maybe 35 URLs one time, and then the next time I run it it will perform fine for perhaps 80 URLs. It never gets to the end of the test list that I'm using which has 137 entries.

I suspect that it may have something to do with pages loading very slowly, so instead of file_get_contents I used fread instead, setting the length parameter to 2000:

Code: Select all

$handle = fopen($url, "r");
			$head = fread($handle, 2000);
			fclose($handle);
This had no effect on the problem.

I also tried to get clever with stream_set_timeout

Code: Select all

$handle = fopen($url, "r");
				
			if (!$handle) {
			   echo "Unable to open\n";
			} else {
			
			   stream_set_timeout($handle, 2);
			   $head = fread($handle, 2000);
			
			   $info = stream_get_meta_data($handle);
			   fclose($handle);
			
			   if ($info['timed_out']) {
			       echo "META TITLE = " . 'Connection timed out!'. "<br><hr>";
			   } else {
			   $title = gettitle($head);
			       echo "META TITLE = " . $title . "<br><hr>";
			   }
But that didn't work either! In fact, none of the error messages show up - the program just reports itself as 'Done' long before it is.

Any ideas? I don't have much hair left and would love to salvage what is still there! :?

Martin


feyd | Please use

Code: Select all

,

Code: Select all

and [syntax="..."] tags where appropriate when posting code. Your post has been edited to reflect how we'd like it posted. Please read:  [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url] to learn how to do it too.[/color]

Posted: Tue Apr 18, 2006 3:14 pm
by feyd
sounds like the script is timing out which has little to do with the requests and more to do with PHP's patience for the script to finish.

set_time_limit()

Posted: Tue Apr 18, 2006 4:36 pm
by Longlands
I've already got set_time_limit(9000) which should be enough for the most impatient script!

The premature arrest of the program seems to be at random times too, so I don't think a fixed time out is the cause.

Martin