High Traffic Website Development

Ye' old general discussion board. Basically, for everything that isn't covered elsewhere. Come here to shoot the breeze, shoot your mouth off, or whatever suits your fancy.
This forum is not for asking programming related questions.

Moderator: General Moderators

Post Reply
rushme
Forum Newbie
Posts: 3
Joined: Mon Dec 27, 2010 12:51 pm

High Traffic Website Development

Post by rushme »

Hi All! Got a best practices/ general type of question regarding developing PHP sites that will have high traffic/page views etc...

I have been developing personal/small business websites for a few years now, but I just started developing a new website that will potentially have millions of page views a month.

I wanted to know if there are any general things that I need to take into account that would be different than when developing for lower traffic websites.

I will be using:
PHP 5
Mysql 5
some javascript libraries (jquery / prototype)

thank you in advance!

rushme
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: High Traffic Website Development

Post by josh »

Stress testing
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: High Traffic Website Development

Post by Weirdan »

In no particular order:
  • profile, then optimize, then profile again
  • number of disk accesses matters (good disk subsystem helps)
  • number of concurrent requests matters (watch out for slow clients)
  • amount of memory consumed by each php process matters (the more memory each process consumes - the fewer concurrent requests you can serve)
  • amount of available memory matters (more memory - more concurrent requests you can serve while still having some memory for disk cache)
  • ... available cpu - not so much usually (for a webserver, anyway)
  • bytecode cache is a must (apc, eaccelerator, xcache - whatever you fancy)
  • client side caching matters (yslow helps)
  • server-side does too (memcache is a popular solution)
  • cache coherency is often an issue - you need to have a way to keep cached data consistent to the data in your persistent storage
  • starting with an empty cache is usually very resource intensive - create a script to prefill most popular cache entries with data. Serve static 'Sorry, we will be back in a moment...' page while you run it.
  • most webservers (even Apache) can serve static files in a resource-efficient way.
  • ... so skip PHP whenever you can and serve a static file
  • webservers specifically tuned for static file serving do that even better
  • ... and you can run it on the same physical box
  • number of network operations matters
  • have backups. keep them offsite as well as onsite. Fires, earthquakes, power outages do happen.
  • ... and have a way to restore to a backed up state quickly (even on a different hosting). Lots of traffic = huge profit loss during downtime.
  • Bad things happen to people as well. Have some staff who can keep you website online. It should not be a single person. They should be able to work from alternative locations (fires, earthquakes and power outages can happen to your office / their homes as well).
  • monitor your website / servers and have an alert system (email/sms). You need to know when something goes awfully wrong. Monitoring system also helps to see positive / negative impact of your optimizations. cacti and nagios are popular solutions that allow you to see semi-realtime statistics using pretty graphs. Pinba is interesting as well.
  • don't assume you'll always have one webserver / one database server. Have (at least) a plan how to use multiple physical database servers and multiple physical webservers.
  • 10M pageviews per day is just a bit more than 100 req/s. With a good software/hardware this number of requests can be easily served by a single web frontend box.
  • don't assume this list to be complete.
rushme
Forum Newbie
Posts: 3
Joined: Mon Dec 27, 2010 12:51 pm

Re: High Traffic Website Development

Post by rushme »

WOW That's a long list :)

I will start by focusing on the following items

1. Choosing the right hosting/server hardware configuration
2. Stress Testing
3. Alert System
4. some sort of redundant system (hardware/software)
5. Daily Backup of data.

I will then turn my attention to proper indexing optimization of my Mysql tables.

Anything to watch out for on the programming side (like not using $_SESSION variables or making sure that certain settings are turned on the apache servers etc...?

Thanks for the feedback! really appreciated.

Rushme
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: High Traffic Website Development

Post by josh »

It can be really useful to have an optional write adapter, in case you are doing master & slave servers for the database. Look into CSS sprite images (combining all images into one image). Look into including all PHP files into one. Look into combining all JS/CSS files into one each. This is really more performance related than high traffic related though. Even magento, one of the slowest scripts, can do 20 pages per second when highly optimized (memcache, nginx, tmpfs, and all the stuff I mentioned about combining PHP, css & JS).

Basically there's page per second, and then there's user load time (performance). The two don't necessarily even correlate.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: High Traffic Website Development

Post by Christopher »

Really it comes down to good caching and a solid multi-server architecture.
(#10850)
rushme
Forum Newbie
Posts: 3
Joined: Mon Dec 27, 2010 12:51 pm

Re: High Traffic Website Development

Post by rushme »

I guess I have to check out a catching solution as well.

I am NOT writing my platform using a templating system (smarty, etc) which would cache my pages. Are there any standalone caching solutions I can hook into, or do I have write one up from the ground up? I've googled a bit... I hit upon Pear::Cache_Lite, but writing my own is an option if no "tried and trusted" off the shelf solutions exists.

thanks!

Rushme
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: High Traffic Website Development

Post by Christopher »

I'd recommend Zend_Cache. It is probably the most current, full-featured and best supported.
(#10850)
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: High Traffic Website Development

Post by josh »

Look into Zend_Cache
Post Reply