Page 1 of 1
replace Goggle Analytics
Posted: Mon Jul 19, 2010 7:05 pm
by Vegan
Given a LAMP box, I was wondering how much headache could there be to make the functionality of Google Analytics with a few lines of JavaScript and a few in PHP to save the results.
Seems JavaScript can read lots of info about the client that could be studied. Recently more handsets are visiting, puts a whole new perspective on design.
Re: replace Goggle Analytics
Posted: Tue Jul 20, 2010 3:48 am
by sam4free
when you write code to collect data you have done only the first 1% of the system.
you need an effective way to store data both in time and storage consumption .. for example:
- Inserting a new record for each page load is a fast way .. but will eat up your hard disk very quickly.
- Updating a row with info (counter or sth) for each page is effective for storage but will consume a lot of CPU time, and you will lose details this way.
Also you will need a code to analyze data and extract the needed information .. this process will be tightly related to the way you store data, and will be mostly executed as a cron job .. you can't just analyze all the data you collected on the fly - when the customer requests a report.
There is a lot of decisions you are gonna be making .. just keep in mind that your system will be dealing with a HUGE amount of data.
Re: replace Goggle Analytics
Posted: Tue Jul 27, 2010 8:33 am
by Vegan
Think 2 TB disk is beg enough for 90 days of data?
Re: replace Goggle Analytics
Posted: Tue Jul 27, 2010 12:57 pm
by jraede
Depends how many people visit your site per day...
Re: replace Goggle Analytics
Posted: Tue Jul 27, 2010 1:03 pm
by Benjamin
2TB is more than enough.
Re: replace Goggle Analytics
Posted: Tue Jul 27, 2010 6:28 pm
by josh
Hint: It doesn't take 2TB of data to store the number "15,000,000". You don't need 15 million rows to represent 15 million hits. Example:
viewtopic.php?f=50&t=112763&start=19
This provides benefit over Google analytics too. Lets say you had users logging in and posting classified ads. You want each user to see only their own traffic, and the admin to see everyones traffic. This isnt possible with analytics but with my framework there is unlimited segmenting and you can use the stats within your application of course. Like I said the key idea if you want to do it yourself is store a cross tabulation. Instead of storing "one one one one one" you'd store "Day 1: 50; Day 2: 1,000", whatever... Pick a "grain". Mine was multi-grained so you could look at traffic for an hour, a day, a month, a year, etc.
So lets say you choose daily. Have a cron job add up the traffic at the end of the day from the "logs" table. Insert one row per statistic into the "daily summary" table. Then you can prune that log table. Your table will grow by 1 row per day that way.
Re: replace Goggle Analytics
Posted: Thu Aug 05, 2010 8:05 pm
by Vegan
Looks like a few good ideas, still working to scrape up some coin to update the server, business has been low this year again.