getting file contents - cron

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
mikesoft
Forum Newbie
Posts: 2
Joined: Sun Aug 08, 2010 3:05 am

getting file contents - cron

Post by mikesoft »

Hey all, again :D

I have this cron job running 4 times a day which basically fetches the contents of an XML file and converts it into an Array with tags as keys. Then, the array values are entered into a database. However, I need to get several XML files, in fact, one for each user existing in my database. And I have to update the database once per user, in the same cron tab.

My code:

Code: Select all

<?php
$result = mysql_query("SELECT id FROM table1"); //getting the ID from the user
while($row = mysql_fetch_array($result)){ 
$contents = file_get_contents("http://www.websitexml.com/feeds/".$row['id']);
$data = XMLtoArray($contents);
mysql_query("UPDATE table1 SET column3='$data[tag1]', column4='$data[tag2]' WHERE id=$row[id]") or die(mysql_error());
}
?>
That code works great, for now. I only need to fetch around 90 feeds (one for each user) which takes about 40 seconds. However, the number of users increases everyday, and I fear that one day there will be so many that the script won't be able to reach the end before hitting the time limit set by my hosting provider. I really don't want to mess with the time limit for script because I don't want my hosting provider to cancel my account, so I want to know if I'm doing it the best it can be done. I know that getting the file contents from another website takes a long time, especially if the other website takes a lot of time to show the data, or if the website itself is slow for some reason, so probably there's nothing I can do about that. But maybe I was thinking of a better way to achieve the same results while taking less time to complete.

If I had to update 500 users right now, I'd probably split the number of users to be updated to about 100, then another script that continues where the last script ended. Honestly I'm not sure how to do that but for now it's not important, the most important thing right now is to know if I'm doing it correctly or if there's a better/faster way.
Post Reply