Page 1 of 1
High Traffic Website Development
Posted: Mon Dec 27, 2010 12:59 pm
by rushme
Hi All! Got a best practices/ general type of question regarding developing PHP sites that will have high traffic/page views etc...
I have been developing personal/small business websites for a few years now, but I just started developing a new website that will potentially have millions of page views a month.
I wanted to know if there are any general things that I need to take into account that would be different than when developing for lower traffic websites.
I will be using:
PHP 5
Mysql 5
some javascript libraries (jquery / prototype)
thank you in advance!
rushme
Re: High Traffic Website Development
Posted: Mon Dec 27, 2010 2:46 pm
by josh
Stress testing
Re: High Traffic Website Development
Posted: Mon Dec 27, 2010 3:09 pm
by Weirdan
In no particular order:
- profile, then optimize, then profile again
- number of disk accesses matters (good disk subsystem helps)
- number of concurrent requests matters (watch out for slow clients)
- amount of memory consumed by each php process matters (the more memory each process consumes - the fewer concurrent requests you can serve)
- amount of available memory matters (more memory - more concurrent requests you can serve while still having some memory for disk cache)
- ... available cpu - not so much usually (for a webserver, anyway)
- bytecode cache is a must (apc, eaccelerator, xcache - whatever you fancy)
- client side caching matters (yslow helps)
- server-side does too (memcache is a popular solution)
- cache coherency is often an issue - you need to have a way to keep cached data consistent to the data in your persistent storage
- starting with an empty cache is usually very resource intensive - create a script to prefill most popular cache entries with data. Serve static 'Sorry, we will be back in a moment...' page while you run it.
- most webservers (even Apache) can serve static files in a resource-efficient way.
- ... so skip PHP whenever you can and serve a static file
- webservers specifically tuned for static file serving do that even better
- ... and you can run it on the same physical box
- number of network operations matters
- have backups. keep them offsite as well as onsite. Fires, earthquakes, power outages do happen.
- ... and have a way to restore to a backed up state quickly (even on a different hosting). Lots of traffic = huge profit loss during downtime.
- Bad things happen to people as well. Have some staff who can keep you website online. It should not be a single person. They should be able to work from alternative locations (fires, earthquakes and power outages can happen to your office / their homes as well).
- monitor your website / servers and have an alert system (email/sms). You need to know when something goes awfully wrong. Monitoring system also helps to see positive / negative impact of your optimizations. cacti and nagios are popular solutions that allow you to see semi-realtime statistics using pretty graphs. Pinba is interesting as well.
- don't assume you'll always have one webserver / one database server. Have (at least) a plan how to use multiple physical database servers and multiple physical webservers.
- 10M pageviews per day is just a bit more than 100 req/s. With a good software/hardware this number of requests can be easily served by a single web frontend box.
- don't assume this list to be complete.
Re: High Traffic Website Development
Posted: Mon Dec 27, 2010 10:46 pm
by rushme
WOW That's a long list
I will start by focusing on the following items
1. Choosing the right hosting/server hardware configuration
2. Stress Testing
3. Alert System
4. some sort of redundant system (hardware/software)
5. Daily Backup of data.
I will then turn my attention to proper indexing optimization of my Mysql tables.
Anything to watch out for on the programming side (like not using $_SESSION variables or making sure that certain settings are turned on the apache servers etc...?
Thanks for the feedback! really appreciated.
Rushme
Re: High Traffic Website Development
Posted: Mon Dec 27, 2010 11:13 pm
by josh
It can be really useful to have an optional write adapter, in case you are doing master & slave servers for the database. Look into CSS sprite images (combining all images into one image). Look into including all PHP files into one. Look into combining all JS/CSS files into one each. This is really more performance related than high traffic related though. Even magento, one of the slowest scripts, can do 20 pages per second when highly optimized (memcache, nginx, tmpfs, and all the stuff I mentioned about combining PHP, css & JS).
Basically there's page per second, and then there's user load time (performance). The two don't necessarily even correlate.
Re: High Traffic Website Development
Posted: Tue Dec 28, 2010 12:33 am
by Christopher
Really it comes down to good caching and a solid multi-server architecture.
Re: High Traffic Website Development
Posted: Tue Dec 28, 2010 9:42 am
by rushme
I guess I have to check out a catching solution as well.
I am NOT writing my platform using a templating system (smarty, etc) which would cache my pages. Are there any standalone caching solutions I can hook into, or do I have write one up from the ground up? I've googled a bit... I hit upon Pear::Cache_Lite, but writing my own is an option if no "tried and trusted" off the shelf solutions exists.
thanks!
Rushme
Re: High Traffic Website Development
Posted: Tue Dec 28, 2010 3:50 pm
by Christopher
I'd recommend Zend_Cache. It is probably the most current, full-featured and best supported.
Re: High Traffic Website Development
Posted: Tue Dec 28, 2010 4:03 pm
by josh
Look into Zend_Cache