Cron vs real-time checking ips

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

Post Reply
jabbaonthedais
Forum Contributor
Posts: 127
Joined: Wed Aug 18, 2004 12:08 pm

Cron vs real-time checking ips

Post by jabbaonthedais »

I'm working on a script for a site that anaylzes incoming traffic. I need it to log all ips and referring urls each day. I know how to do all the coding for the script. My question has to do with speed. I need to get each user's ip and referring url, check it against the list of other ip's/urls of that day, and log the uniques to a table. How slow/resource intensive is this likely to be on the server? I could do it real-time or make a cron run every few minutes... but wouldn't that be the same result? When the cron ran, the server is going to be using the same resources, just not spread out. I want the stats to be semi-real-time, so a user can check the stats at any point during the day. So if I did have a cron, it would have to run at least once every 10 minutes.

I know this would be easy for a site with little traffic. But if you had 25k hits a day... I wonder how long it would take a sql query to determine if the ip is unique with 25k ip's in the table.

I know many scripts do similar things but not sure exactly how they go about it... Any thoughts or suggestions?
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Post by John Cartwright »

If your talking about lots of datahits on the database, I would definantly go with a cron. More so, what I've done in the past is generate static files .. atlhough this was some pretty statistical and rescource intensive stuff. There are a couple advantages and disadvantages for going whichever way, however it really depends on your personal preference. As I've said before, I'd go with the cron for a couple reasons. It seperates the logic between two scripts (retrieving and updating results), reduces user load times, and could spread the load a bit (although that is purely speculation).

Just to recap quickly, my preference is cron + static file creation if speed is among the most important factors.
jabbaonthedais
Forum Contributor
Posts: 127
Joined: Wed Aug 18, 2004 12:08 pm

Post by jabbaonthedais »

Speed is definately the important factor here. The only question I have though... isn't a cron just focusing the load on a few users vs. spreading it out over many users? Maybe I'm overthinking this... but the way I see it, my cron is going to be doing the same amount of server load as me doing the script for each individual user. Maybe I'm wrong and putting many ips together in an array will be faster than checking each ip seperately?
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Post by John Cartwright »

jabbaonthedais wrote:Speed is definately the important factor here.
Gotcha.
jabbaonthedais wrote:isn't a cron just focusing the load on a few users vs. spreading it out over many users?
By using a cron, you are focusing the load on none of your users. Cron is behind the scenes compared to a user visiting a page and triggering the processing.
jabbaonthedais wrote:my cron is going to be doing the same amount of server load as me doing the script for each individual user.
It really all depends on your situation (which I know next to nothing about). In which case, I'll use my work as an example. We would have about 50 different sites and we needed to track all the user activity corresponding to each site. Now, lets say I log into my admin page to check these stats, if every page load I wanted to recalculate all the statistics I would have very load page load times. Now, since we used a cron, things were kept super fast. The data accumulated over 5 minutes (cron cycle) is usually much less than lets say 72 hours (maybe we didnt visit in last 72 hours).
jabbaonthedais wrote:Maybe I'm wrong and putting many ips together in an array will be faster than checking each ip seperately?
Not sure what you mean. :? Again, you have to remember we don't really know what your doing.
jabbaonthedais
Forum Contributor
Posts: 127
Joined: Wed Aug 18, 2004 12:08 pm

Post by jabbaonthedais »

Thanks Jcart for the great reply!

What I meant when I said "focusing the load on one user" was that any user visiting at the time of the cron would be affected by the a sever load as opposed to each user taking a piece of that load. Though its obviously depends on your server and how much bandwidth/resources you have.

And about the ips in an array... If I checks ip's each time a user visits, I would run a query for the user's ip and see if it matched a ip in "today"'s table. If it didn't match, it would create a new row and add the user's ip. I'm not sure if using an array of ip's in a cron (the last 5-10 minutes of ip's) would be any faster than one ip at a time, was basically what I was getting at.
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Post by John Cartwright »

In that case I would avoid using a cron considering it is only a simple query (if your using mysql)

Code: Select all

$sql = 'REPLACE `ip_table` 
SET `ip` = \''. mysql_real_escape_string($_SERVER['REMOTE_ADDR']).'\'
WHERE `time` > DATE_SUB(CURDATE(),INTERVAL 1 MONTH)';
If you used a cron, you'd still need to insert the ip into the cron queue and THEN check if it exists in the db. The statistics behind this are as simple as it gets, no need to overcomplicate things.
jabbaonthedais
Forum Contributor
Posts: 127
Joined: Wed Aug 18, 2004 12:08 pm

Post by jabbaonthedais »

lol, I didn't even think about having to log it for the cron. Thanks Jcart! Your advice has greatly helped me. :)
Post Reply