Page 1 of 1
Visitor Statistics
Posted: Mon Feb 18, 2008 11:48 am
by ed209
Hi,
I have a multi user site and I'd like to add statistics for each of the users. Each time someone's profile is visited I'd like to record
-time/date
-user id (if set)
-some other relevant data
It wouldn't just be recording visits to the profile page, it could be any page linked to that user.
Where would I go about storing that data? The easiest way would be to write a row to a DB but I'm not sure about the long term if there are millions of rows. Are there any open source solutions that you know about that could handle this kind of functionality? Should I serialize the data to a file?
Re: Visitor Statistics
Posted: Mon Feb 18, 2008 3:23 pm
by califdon
How do you expect to use the data? It sounds obvious to me that it should be in a database, even more so if the number of entries will grow to a large number.
Re: Visitor Statistics
Posted: Mon Feb 18, 2008 3:38 pm
by Sekka
I agree with califdon, database storage seems the best solution. Occam's Razor!
But, it depends on what the data is used for. If it is totalling, you may want to process the logs and store totals somewhere else so you don't need to constantly access the logs themselves? Saves time and processing?
Re: Visitor Statistics
Posted: Mon Feb 18, 2008 4:15 pm
by ed209
stats will be used for creating simple bar charts, very basic user numbers over time etc. This is a non tech community so it doesn't need to be too in-depth. What data do you find most useful that's easy to store/get? Here's what I'm thinking:
datetime
page viewed (profile page etc)
user id (did another logged in website member browse your profile?)
repeat visitor / unique id (stored as cookie, any other way of doing this?)
Location (any way to get which country they are from?)
That's probably about it, do you think there is any other data that non-tech people might find interesting?
Thanks for your help.
Re: Visitor Statistics
Posted: Mon Feb 18, 2008 4:55 pm
by califdon
ed209 wrote:stats will be used for creating simple bar charts, very basic user numbers over time etc. This is a non tech community so it doesn't need to be too in-depth. What data do you find most useful that's easy to store/get? Here's what I'm thinking:
datetime
page viewed (profile page etc)
user id (did another logged in website member browse your profile?)
repeat visitor / unique id (stored as cookie, any other way of doing this?)
Location (any way to get which country they are from?)
That's probably about it, do you think there is any other data that non-tech people might find interesting?
Thanks for your help.
In a database, the usual approach would be to store each visit as a record in a table, then you can determine repeat visitors with the query when you create the stats. That provides the flexibility to accumulate summaries over differing periods of time.
As
Sekka said, you might want to consider extracting summaries at intervals (monthly or whatever), from which you could generate your charts. I would advise retaining the original data, however, just for such questions as repeat visitors spanning multiple time intervals. Databases are just fine for retaining millions of records.
As for geographic origins, there are several approaches. Fundamentally, domain names and even IP addresses are not accurate determiners of true locations, because anyone can subscribe to an account somewhere other than where they actually are. But that may not be critical for your purposes. Generally, using the domain will identify only some of the countries, since some foreign companies will probably have .com or .net domains. There are now several services available (some of them free, even!) that have databases of IP address blocks and can determine geographic location to within perhaps 100 miles or less for
most IP addresses. Even this doesn't hold true for AOL subscribers, though, for example. I use this one on a couple of my websites:
http://geobytes.com/. Check out my log at
http://usselectra.org/showvisitslog.php and see how I use it at
http://usselectra.org, based on essentially free interface from Geobytes. Another of my sites is
http://poatree.org, where you can see that I have plotted your approximate location on a map. Lots of fun things you can do with geolocation!

Re: Visitor Statistics
Posted: Tue Feb 19, 2008 2:43 am
by ed209
thanks for the information, location stuff looks great
regarding the unique visitor, I was just wondering what data I would store in the database in order to mark them as unique. I was thinking along two lines, firstly creating some unique value and storing it as a cookie and then recording that unique cookie value against each row. That way I could trace the unique visitors rather than unique visits. I would use IPs but most broadband ISPs assign dynamic IPs.
The other option would be to record the session id, but that would only log unique visits as opposed to visitors, which in this case may be good enough.
Finally, regarding the location, should I calculate / store the visitor location on writing to the DB or store the information used to calculate the location for processing on requesting the stats / processing the total for storage?
Re: Visitor Statistics
Posted: Tue Feb 19, 2008 11:15 am
by califdon
ed209 wrote:regarding the unique visitor, I was just wondering what data I would store in the database in order to mark them as unique. I was thinking along two lines, firstly creating some unique value and storing it as a cookie and then recording that unique cookie value against each row. That way I could trace the unique visitors rather than unique visits. I would use IPs but most broadband ISPs assign dynamic IPs.
I don't think you can really get around that. It's fundamental to the arbitrary "identification" situation. There is just no way for you to know that it's the same person on a subsequent visit unless you implement a user login.
Finally, regarding the location, should I calculate / store the visitor location on writing to the DB or store the information used to calculate the location for processing on requesting the stats / processing the total for storage?
It's possible to do it either way, since GeoBytes and others provide a way to analyze entire logfiles, but I don't like to count on an external source for such things, so once I have a location, I log that.