Page 1 of 2

Cronjob every minute?

Posted: Sat Apr 29, 2006 6:43 am
by s.dot
Is it impractical to run a simple (very simple) cron job every minute?

I need to display an online count, but I don't want to query several thousand user rows on every page load. Instead could I store this 1 number in a database table and call that instead? What are the downsides to running a cronjob every minute? Is it common practice?

Re: Cronjob every minute?

Posted: Sat Apr 29, 2006 6:48 am
by Chris Corbyn
scottayy wrote:Is it impractical to run a simple (very simple) cron job every minute?

I need to display an online count, but I don't want to query several thousand user rows on every page load. Instead could I store this 1 number in a database table and call that instead? What are the downsides to running a cronjob every minute? Is it common practice?
You could have your users generate the count once-only. Basically, when you store the count, store a timestamp with it. When a user visits the page, if the timestamp is more than a minute since the last time update the count, otherwise, just use the one that's already stored.

But to answer your question I wouldn't see an issue with running a simple cron task once a minute... cron is designed for things like this ;)

Posted: Sat Apr 29, 2006 6:58 am
by timvw
I've got the feeling you're in the following situation: When your user performs an action, you store the time.. And then you want to show something like: it has been X minutes since your last action.. This results in an enormous overhead..

Why don't you calculate the difference when it's needed? The least intrusive solution would be a view that calculates the value for the 'difference' column as the difference between now and the value in the 'last_action_time'

Posted: Sat Apr 29, 2006 5:41 pm
by s.dot
Good guess :P, but its a bit different.
I'm displaying my site's stats at the top of every page. IE Members: xxx Online: xxx Guests: xxx
Counting those rows on every pageload is slowing down pageload times.

Posted: Sat Apr 29, 2006 6:55 pm
by John Cartwright
I've got a project where I need to analyse realtime some stats, so what do you guys actually think of running a cron every minute? Perhaps even every 30 seconds.
Having built in code to each website is not possible as this data is all sent to a master server.

Thanks.

Posted: Sat Apr 29, 2006 7:05 pm
by Christopher
scottayy wrote:I'm displaying my site's stats at the top of every page. IE Members: xxx Online: xxx Guests: xxx
Why don't you do what Google and Yahoo do and have your cron write to a tiny HTML file that you include in you pages. Then there is no query to read and only once a minute to write the 100 byte file.

Posted: Sat Apr 29, 2006 7:16 pm
by n00b Saibot
If you want realtime then it has to be 0.0001 seconds atleast.

Posted: Sat Apr 29, 2006 7:29 pm
by Burrito
in all seriousness...I don't see any issues with running a cron every 15 or even 5 seconds. Essentially it's just one hit on the server and depending on how intensive the job is (for this topic, wouldn't need to be intensive at all) that's pennys...

Posted: Sat Apr 29, 2006 7:52 pm
by santosj
Um, maybe I'm missing something, but um what about this:

1. Add timestamp to the user table.

Code: Select all

mysql_query('UPDATE user_table SET last_active='. time() ." WHERE user='". mysql_escape_string($_SESSION['user']) ."'");

$query = mysql_query('SELECT COUNT(user) FROM user_table WHERE last_active=last_active > '. strtotime("-1 minute"));

$count = 0;
if($query)
{
	$fetch = mysql_fetch_array($query, MYSQL_NUM);
	$count = $fetch[0];

	// $fetch = mysql_fetch_assoc($query);
	// $count = $fetch['COUNT(user)'];
}

echo $count;

Posted: Sat Apr 29, 2006 10:40 pm
by s.dot
To the post above me, I don't want to hit the server with this query every page load.

But on another note, I can't seem to get my cron working.

I am using the cPanel interface, and I set it to this

* * * * * (hour minute day weekday ..something else)

command: /home/local/bin/php /smp/public_html/admin/cron_membercount.php

Doesn't seem to be emailing me the results or doing the job.

What's the equivalent to this on command line?

Posted: Sat Apr 29, 2006 10:50 pm
by s.dot
arborint wrote:
scottayy wrote:I'm displaying my site's stats at the top of every page. IE Members: xxx Online: xxx Guests: xxx
Why don't you do what Google and Yahoo do and have your cron write to a tiny HTML file that you include in you pages. Then there is no query to read and only once a minute to write the 100 byte file.
That is a really great idea.

Posted: Sun Apr 30, 2006 8:08 am
by santosj
I'm sorry, but running a cron job every minute is just as bad as running that with every page load. The point of using SQL is that it is quicker than running the cron job. What if your site isn't receiving any traffic at all? Are you just going to keep running and wasting resources on a cron job? If you are using Smarty or Template Lite, or any templating system with caching, then the solution is built in without any need to hit the MySQL server or using a cron job.

But I'm a man of solutions and here is one.

Code: Select all

$count = 0;
if(!isset($_SESSION['last_update']) or ($_SESSION['last_update'] < strtotime("-1 minute"))
{
	$_SESSION['last_update'] = time();
	// Above Mysql data
	$SESSION['members'] = $array[0];
}
else
{
	$count = $SESSION['members'];
}

echo $count;
Now this code will only hit the server every minute instead of every page load.

Here is what you do:

1. Create an array that holds, the guests, members, and administrators online.
2. Serialized that array
3. Output the array to a file with file_put_contents()
4. Use filemtime() to get the time of which it was modified.
5a. If it was modified under a minute ago, then grab the contents.
6a. Unserialized the array from the file with file_get_contents().
7a. Populate the main array with the array from the file.

5b. If the file was modified over a minute ago, then ignore the contents.
6b. Grab the information from the mysql database.
7b. Repeat step 2 and 3
8b. Populate the main array.

Cron Job

From your specifications you have stated that you wouldn't mind hitting the mysql database within a new table. I don't see how it is any different from my example, except it is smarter.

But lets for example say that you wrote it to a file instead (better idea for small amounts of data). On the off hours you would run the script for no reason at all running transactions to the database for no reason.

Mysql Transaction on every request

I really don't see the point of why this is such an god awful idea. I do on most of the scripts I write and unless you are querying the database for every little thing, I don't see how the overhead would effect the over processing time. You can optimize the SQL further. When you said you didn't want to query every row, I thought you were using a While loop, so my bad for misunderstanding your level of skillz.

This is by far the easiest, simple method and I think you underrate it just a little bit. "No KISSing for you Mr Santos, you sick freak you!"

My Second Example

Benefits from both worlds of the cron job and my first method. The complexity goes through the roof, but should only take less than 15 minutes to implement if you are a l33t coder or have used the functions before.

Since the cron job isn't working, use my second method. If you can't (or don't) want to code it, then I can provide the code for you, but I will probably do it based on what I know and it could have some bugs that would need to be worked out. As I have never done any serialization, you'll have to excuse that bit... but seriously, how hard can it be? PHP does all the work for you.

Posted: Sun Apr 30, 2006 1:25 pm
by Christopher
santosj wrote:I'm sorry, but running a cron job every minute is just as bad as running that with every page load. The point of using SQL is that it is quicker than running the cron job. What if your site isn't receiving any traffic at all? Are you just going to keep running and wasting resources on a cron job?
How can you be "wasting resources" by running a trivial script on a server with no load?!? If anything, added intellegence to the script would check of the load on the server was above a certain level and not run under those conditions. That's because page response time is more important than super accurate stats that few people ever read.

I have found in practice that crons add much less load than solutions in the appliation for problems such as these.

Posted: Sun Apr 30, 2006 1:27 pm
by Burrito
arborint wrote:How can you be "wasting resources" by running a trivial script on a server with no load?!? If anything, added intellegence to the script would check of the load on the server was above a certain level and not run under those conditions.

I have found in practice that crons add much less load than solutions in the appliation for problems such as these.
I'll second that. Let's assume that your site gets 1000 hits a minute, if you include your script on your page, you're talking 1000x to load that script vs 1 every 5 minutes or 1 every minute for that matter....it's not at all the same.

Posted: Sun Apr 30, 2006 6:42 pm
by s.dot
SantosJ, your code sample does indeed hit the server with a query on every page load; which would be what I'm to avoid. :)