Page 1 of 1

Scaling PHP

Posted: Sun Aug 09, 2009 12:44 am
by volomike
I compiled a list of steps I think would be good to follow for scaling PHP. Please reply if you think the order could be different, or if you have other ideas.

On most of this, you'll probably want to start right away and apply all the steps up to F and then stop for awhile, only going past that when you think your page load times on certain pages require it.

A. Improve the PHP code.
B. Improve the SQL queries and index configuration.
C. Tweak the config of Apache, MySQL or PostgreSQL, and PHP so that they are not on the defaults.
D. Utilize APC in the PHP and also use an APC cache object for caching things you repeatedly use.
E. Add page compression and output buffering to your .htaccess file.
F. Load your images from a separate domain pointing to the same web space -- it overcomes a browser limitation and improves page loading speed.
G. (Consider moving from MySQL to PostgreSQL.)
H. Install database tools to help you analyze your index config and your database to see if you can retune it, and combine this with the various free forum and paid support channels.
I. Add more RAM.
J. Add more CPU power.
K. Install Munin and NetSNMP to chase bottlenecks. Use free forum and paid support channels to resolve issues where possible. Don't overdo this though -- sometimes it's easier to just add more RAM, CPU, or move up the chain here in steps.
L. Rewrite some components in C to help alleviate some bottlenecks. (Ask me if interested.) Don't overdo it, though -- just focus on the bottlenecks and the low-hanging fruit.
M. Build a simple web farm using Apache mod_proxy (ask me if interested) with one beefy head server (also acting as database server) and 2 web nodes. (Consider VPS hosting for this.)
N. Add more VPS web nodes.
O. Move from VPS hosting to Dedicated hosting -- and at least on the head server, if not the nodes as well. It's going to be expensive.
P. Keep adding web nodes.
Q. Create a single write-master database config with multiple read-masters. That way, multiple databases can respond with the exact same data on reading data, but on writing data it can go to one central connection.
R. Replace the Apache mod_proxy web farm with a Cisco Local Director -- it's expensive.
S. Get all the networking on fiber optics and between multiple pizza box servers all on the same subnet.
T. Get a Cisco routing expert to improve performance.
U. Bring in the big dogs, such as a performance guru on the hardware you're using.
V. Move from a single database system into a 3 database system. One database is normalized heavily and is used for write-only transactions. These then get written to the main database either on the hour, every 2 hours, every 4 hours, etc. -- whatever you desire and can handle. Database two is used as your central, main database where you guarantee your data is verified and is used for read/write transactions, such as record editing. The main database is usually normalized just a little and usually only contains just a couple years of data. Database three is your historical/reporting database (it serves both purposes) and is heavily denormalized so that report queries can run very fast. The historical/reporting database contains all your archived data from yesterday and back as far as, oh, 15 years let's say -- whatever you need. So, every day, a nightly job makes a copy of the day's worth of validated data and pushes it to the historical/reporting database. This configuration essentially improves read and write performance for your website.

Re: Scaling PHP

Posted: Sun Aug 09, 2009 3:47 am
by Eran
Step number one and number two should be profiling. Before you start doing anything, first understand where your bottlenecks are, instead of optimizing blindly. You should continue to profile after every step as well. Following this, the first step to scaling would be to install those monitoring / profiling solutions (such as munin, xdebug etc)

Re: Scaling PHP

Posted: Sun Aug 09, 2009 11:54 pm
by volomike
pytrin wrote:...should be profiling...
I agree with your opinion.

Re: Scaling PHP

Posted: Mon Aug 10, 2009 1:02 am
by Christopher
It seem like A-L are tuning and not scaling. M-R are about building a mod_proxy based system. R-U are about upgrading to a Cisco based system. And V is general synopsis.

It seem like this would be more useful to people here if you started with a mod_proxy and worked backward to show people what needs to be changed in their application to work with it. Moving to Cisco could be done as necessary because the application capabilities would already be in place.

Re: Scaling PHP

Posted: Mon Aug 10, 2009 8:06 am
by volomike
For those interested in this thread, you might also be interested in this one:

viewtopic.php?f=14&t=104414

Re: Scaling PHP

Posted: Mon Aug 10, 2009 11:41 am
by Christopher
I assume that is for your item L above...