Hi,
I have a multi user site and I'd like to add statistics for each of the users. Each time someone's profile is visited I'd like to record
-time/date
-user id (if set)
-some other relevant data
It wouldn't just be recording visits to the profile page, it could be any page linked to that user.
Where would I go about storing that data? The easiest way would be to write a row to a DB but I'm not sure about the long term if there are millions of rows. Are there any open source solutions that you know about that could handle this kind of functionality? Should I serialize the data to a file?
Visitor Statistics
Moderator: General Moderators
Re: Visitor Statistics
How do you expect to use the data? It sounds obvious to me that it should be in a database, even more so if the number of entries will grow to a large number.
- Sekka
- Forum Commoner
- Posts: 91
- Joined: Mon Feb 18, 2008 10:25 am
- Location: Huddersfield, West Yorkshire, UK
Re: Visitor Statistics
I agree with califdon, database storage seems the best solution. Occam's Razor! 
But, it depends on what the data is used for. If it is totalling, you may want to process the logs and store totals somewhere else so you don't need to constantly access the logs themselves? Saves time and processing?
But, it depends on what the data is used for. If it is totalling, you may want to process the logs and store totals somewhere else so you don't need to constantly access the logs themselves? Saves time and processing?
Re: Visitor Statistics
stats will be used for creating simple bar charts, very basic user numbers over time etc. This is a non tech community so it doesn't need to be too in-depth. What data do you find most useful that's easy to store/get? Here's what I'm thinking:
datetime
page viewed (profile page etc)
user id (did another logged in website member browse your profile?)
repeat visitor / unique id (stored as cookie, any other way of doing this?)
Location (any way to get which country they are from?)
That's probably about it, do you think there is any other data that non-tech people might find interesting?
Thanks for your help.
datetime
page viewed (profile page etc)
user id (did another logged in website member browse your profile?)
repeat visitor / unique id (stored as cookie, any other way of doing this?)
Location (any way to get which country they are from?)
That's probably about it, do you think there is any other data that non-tech people might find interesting?
Thanks for your help.
Re: Visitor Statistics
In a database, the usual approach would be to store each visit as a record in a table, then you can determine repeat visitors with the query when you create the stats. That provides the flexibility to accumulate summaries over differing periods of time.ed209 wrote:stats will be used for creating simple bar charts, very basic user numbers over time etc. This is a non tech community so it doesn't need to be too in-depth. What data do you find most useful that's easy to store/get? Here's what I'm thinking:
datetime
page viewed (profile page etc)
user id (did another logged in website member browse your profile?)
repeat visitor / unique id (stored as cookie, any other way of doing this?)
Location (any way to get which country they are from?)
That's probably about it, do you think there is any other data that non-tech people might find interesting?
Thanks for your help.
As Sekka said, you might want to consider extracting summaries at intervals (monthly or whatever), from which you could generate your charts. I would advise retaining the original data, however, just for such questions as repeat visitors spanning multiple time intervals. Databases are just fine for retaining millions of records.
As for geographic origins, there are several approaches. Fundamentally, domain names and even IP addresses are not accurate determiners of true locations, because anyone can subscribe to an account somewhere other than where they actually are. But that may not be critical for your purposes. Generally, using the domain will identify only some of the countries, since some foreign companies will probably have .com or .net domains. There are now several services available (some of them free, even!) that have databases of IP address blocks and can determine geographic location to within perhaps 100 miles or less for most IP addresses. Even this doesn't hold true for AOL subscribers, though, for example. I use this one on a couple of my websites: http://geobytes.com/. Check out my log at http://usselectra.org/showvisitslog.php and see how I use it at http://usselectra.org, based on essentially free interface from Geobytes. Another of my sites is http://poatree.org, where you can see that I have plotted your approximate location on a map. Lots of fun things you can do with geolocation!
Re: Visitor Statistics
thanks for the information, location stuff looks great
regarding the unique visitor, I was just wondering what data I would store in the database in order to mark them as unique. I was thinking along two lines, firstly creating some unique value and storing it as a cookie and then recording that unique cookie value against each row. That way I could trace the unique visitors rather than unique visits. I would use IPs but most broadband ISPs assign dynamic IPs.
The other option would be to record the session id, but that would only log unique visits as opposed to visitors, which in this case may be good enough.
Finally, regarding the location, should I calculate / store the visitor location on writing to the DB or store the information used to calculate the location for processing on requesting the stats / processing the total for storage?
regarding the unique visitor, I was just wondering what data I would store in the database in order to mark them as unique. I was thinking along two lines, firstly creating some unique value and storing it as a cookie and then recording that unique cookie value against each row. That way I could trace the unique visitors rather than unique visits. I would use IPs but most broadband ISPs assign dynamic IPs.
The other option would be to record the session id, but that would only log unique visits as opposed to visitors, which in this case may be good enough.
Finally, regarding the location, should I calculate / store the visitor location on writing to the DB or store the information used to calculate the location for processing on requesting the stats / processing the total for storage?
Re: Visitor Statistics
I don't think you can really get around that. It's fundamental to the arbitrary "identification" situation. There is just no way for you to know that it's the same person on a subsequent visit unless you implement a user login.ed209 wrote:regarding the unique visitor, I was just wondering what data I would store in the database in order to mark them as unique. I was thinking along two lines, firstly creating some unique value and storing it as a cookie and then recording that unique cookie value against each row. That way I could trace the unique visitors rather than unique visits. I would use IPs but most broadband ISPs assign dynamic IPs.
It's possible to do it either way, since GeoBytes and others provide a way to analyze entire logfiles, but I don't like to count on an external source for such things, so once I have a location, I log that.Finally, regarding the location, should I calculate / store the visitor location on writing to the DB or store the information used to calculate the location for processing on requesting the stats / processing the total for storage?