Page 1 of 1

cycle through this in sections

Posted: Sun Feb 12, 2006 7:34 pm
by jasondavis
I have the code for my script below, what it does is cycles through a lot of entries in the DB and Updates the data fro them, the problem is I need to have it do like 5 at a time then do 5 more then 5 more and so on instead of doing them all at the same time which makes the script timeout





Code: Select all

<html>
<body>

<?php
include "globals.php";
include "db.php";

	$query = "SELECT intMSId FROM users WHERE trainnumber='$trainnumber' AND intPriority > 0";
	$res = mysql_query($query) or die("Error: " . mysql_error());
	$count = 0;
	while ($row = mysql_fetch_array($res)) {
		$MSid = $row['intMSId'];

		$html = file_get_contents("http://profile.myspace.com/index.cfm?fuseaction=user.viewprofile&friendID=$MSid");
		preg_match("/<span class=\"nametext\">(.*)<\/span>/", $html, $msname);
		preg_match("/<td class=\"text\" width=\"193\" bgcolor=\"#ffffff\" height=\"75\" align=\"left\">\"(.*)\"<br>/", $html, 	$msdata);
		preg_match("/has <span class=\"redbtext\">(\d+)<\/span> friends/", $html, $msfc);
		preg_match("/(\d+) years old/", $html, $msyo);
		preg_match("/ctl00_Main_ctl00_.*<img src=\"(.*)\" border=\"0\" \/><\/a>/", $html, $msimg);
				
		$msdata[1] = mysql_escape_string($msdata[1]);
		$msname[1] = mysql_escape_string($msname[1]);

		if ($msname[1] != "")
		{
			$query = "update users set txtMSName='"
				. addslashes($msname[1]) . "', intMSAge='"
				. addslashes($msyo[1]) . "', intMSFriends='"
				. addslashes($msfc[1]) . "', txtMSText='"
				. addslashes($msdata[1]) . "', txtMSImg='"
				. addslashes($msimg[1]) . "' WHERE intMSId=$MSid";
			mysql_query($query) or die("Error: " . mysql_error());

			$count++;
		} else
		{
			$error = "User doesnt exists ($MSid)";
			print "$error<br>";
		}
	}
	echo "<p>$count users with priority (> 0) have been updated.</p>";
?>

</body>
</html>

Posted: Sun Feb 12, 2006 7:44 pm
by neophyte
Increase the time allowed for execution in the script. Something like:

Code: Select all

ini_set('max_execution_time', '300');
I think that's the right ini variable.

Posted: Sun Feb 12, 2006 7:51 pm
by Benjamin

Code: Select all

if ($_GET['executeagain'] == true) {
  for ($i = 1; $i <= 5; $i++) {
    // your code here
    header (Location http://mydomain.com/myscript.php?executeagain=true);
  }
}
edit: forgot code tags

Posted: Sun Feb 12, 2006 7:52 pm
by jasondavis
I may be wrong but lets say I have 100 entries in the DB and when I run this script it is grabbing data off of 100 myspace profiles, thgat cant be good for the server can it?

Posted: Sun Feb 12, 2006 8:01 pm
by jasondavis

Code: Select all

<?php
include "/home/myspace/public_html/10k/inc/globals.php";
include "/home/myspace/public_html/10k/inc/db.php";

if ($_GET['executeagain'] == true) { 
  for ($i = 1; $i <= 5; $i++) { 

	$query = "SELECT intMSId FROM users WHERE trainnumber='$trainnumber' AND intPriority > 0";
	$res = mysql_query($query) or die("Error: " . mysql_error());
	$count = 0;
	while ($row = mysql_fetch_array($res)) {
		$MSid = $row['intMSId'];

		$html = file_get_contents("http://profile.myspace.com/index.cfm?fuseaction=user.viewprofile&friendID=$MSid");
		preg_match("/<span class=\"nametext\">(.*)<\/span>/", $html, $msname);
		preg_match("/<td class=\"text\" width=\"193\" bgcolor=\"#ffffff\" height=\"75\" align=\"left\">\"(.*)\"<br>/", $html, 	$msdata);
		preg_match("/has <span class=\"redbtext\">(\d+)<\/span> friends/", $html, $msfc);
		preg_match("/(\d+) years old/", $html, $msyo);
		preg_match("/ctl00_Main_ctl00_.*<img src=\"(.*)\" border=\"0\" \/><\/a>/", $html, $msimg);
				
		$msdata[1] = mysql_escape_string($msdata[1]);
		$msname[1] = mysql_escape_string($msname[1]);

		if ($msname[1] != "")
		{
			$query = "update users set txtMSName='"
				. addslashes($msname[1]) . "', intMSAge='"
				. addslashes($msyo[1]) . "', intMSFriends='"
				. addslashes($msfc[1]) . "', txtMSText='"
				. addslashes($msdata[1]) . "', txtMSImg='"
				. addslashes($msimg[1]) . "' WHERE intMSId=$MSid";
			mysql_query($query) or die("Error: " . mysql_error());

			$count++;
		} else
		{
			$error = "User doesnt exists ($MSid)";
			print "$error<br>";
		}
	}
	echo "<p>$count users with priority (> 0) have been updated.</p>";

  header (Location http://mysite.com/10k/admin/update.php? ... again=true); 
  } 
} 
?>

I tried this but get

Parse error: parse error, unexpected T_STRING in /home/myspace/public_html/10k/admin/update.php on line 43

Posted: Sun Feb 12, 2006 8:03 pm
by Benjamin
It's good practice not to hit a server more than once every 2 seconds.

You will need to modify your code to place in the for loop otherwise it will try to do the same thing it's already doing 5 times.

Posted: Sun Feb 12, 2006 8:04 pm
by Benjamin
agtlewis wrote:

Code: Select all

header ("Location http://mydomain.com/myscript.php?executeagain=true");
I forgot the qoutes in the header()

Posted: Sun Feb 12, 2006 8:20 pm
by John Cartwright
sleep() may be useful aswell

Posted: Mon Feb 13, 2006 6:42 pm
by jasondavis
I dont have enough experience to get this to work, would anyone be willing to help me out for money through papyal?

Posted: Mon Feb 13, 2006 7:12 pm
by josh
If you're willing to pay someone to do that why not pay for a decent host that doesn't impose time limits on your scripts without letting you change it as needed?

Posted: Mon Feb 13, 2006 7:25 pm
by jasondavis
jshpro2 wrote:If you're willing to pay someone to do that why not pay for a decent host that doesn't impose time limits on your scripts without letting you change it as needed?
I guess im just looking for a way to lessen the server load because the script uses file_get_contents and it is going through like 100 entry using file_get_contents for each one, its a lot to run on one page load so Im just wanting to split it up, I once had a member script that would email members and since there was thousands of members it would do like 20 then 20 more then 20 more and so on