Scalability

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Scalability

Post by Christopher »

Weirdan wrote:And it's not sharding, just ordinary M-M setup, because there are no disconnected shards anymore since you started to replicate (in both directions, I suppose).
I didn't actually replicate back in my implementation. I think I just mentioned that as an option. One the project they knew they were going to gather a very large number of records, and knew about how many they end up with eventually, so we provisioned accordingly. My point was not that it was a flexible or powerful implementation. My point was it was a very, very simple solution.
(#10850)
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Scalability

Post by Christopher »

William wrote:The problem with that is it's limited to a set number of servers. My two cents on those whole project is scaling small-medium sites is not that difficult.

(These are examples)
Step #1
Master/Master - ( Just two servers, keep them at around 50% load each that way either master can take over if something goes wrong )

Step #2 (you grow)
Master/Master with slaves. You read from slaves, you write to masters, simple.

Step #3
As you grow more and more reads you can start doing things like have slave-masters that other slaves will replicate off it's slave. So lets say the master writes to 10 slaves, then those 10 slaves write to 10 slaves each, you can scale reads forever almost.

Step #4
Okay so you're starting to have write issues. There is tons of ways to solve this problem as specified in posts before. Horizontal Sharding ( e.g. Splitting table data across multiple servers. For instance you might have 1,000,000 users on server A while the other 500,000 is on server B. ) Vertical Sharding ( e.g. Splitting up table(s) across multiple databases. You can do something like putting the entire users table on a separate database server, or before you have that big of a users table you can put multiple tables on the same database server. Basically instead of having one huge database server, you cut the tables into groups.)
I want to thank you William, this is exactly the kind of practical information I was looking for. So you think Master/Master is the next logical step? I really like the progression you present. It gives clear steps from 1 to 2 to 3 to N servers. I think it would be more than enough for most of our sites that need a little scalability and higher availability. Probably at some point, for some configurations, a cluster will be a better solution.

What do you think about active/passive masters? It is sort of a minimal version of #1 with one read/write server and one read server.

What sort of application support is needed for master/slave architectures? I know that many projects have implemented database code like WordPress's HyperDB to support these architectures.

And, what sort of information is useful in setting up and managing master and slave server, and bring failed server back online?
William wrote:Now the problem with this comes into relationships. You have to start denormalizing your data. JOINS are not your friend as you scale. For instance, Facebook doesn't have a single JOIN in production. To solve issues like this sites like Flickr, Facebook, Digg, etc start replicating their data across their servers. For instance User ID 51789 might be on Server #482 and they'll not only store that users information, but any comments, picture entries, etc on that server also. If two users needs to have the same data joined, they'll replicate the data. For instance an entry in the database links to two different users, then each of the user's servers will have that data. Sure, denormalization makes things much harder as your application logic needs to start adding in the fact that you duplicate data and need to update both, etc. But no one said this would be easy.

Also, to fix the issue with not being able to have primary keys, a lot of big sites have index servers. These index servers contain information tow here all the data is at. For instance I'll connect to my users index and ask them where user 45 is. It will return what cluster / server that user is located and then I'll connect to that.
I am not sure this stuff is too practical to most programmers here, but it is interesting. It might be interesting to know which if these techniques are useful for smaller solutions.
William wrote:Scaling isn't just the database, you can reduce A LOT of load on your database servers by caching. For instance, Memcache. Facebook has a 99% cache hit ratio. That means that 99% of the time they can pull data off their cache instead of having to get data from their database. That helps a lot. Even simple things like fragment caching can be amazing, on your main page for instance you can set data to expire after 4 hours. Or you can make it not expire at all and in the backend whatever is updating that info, also updates the cache. It's really simple.

For scaling static content, you can use a 3rd party service like Amazon S3, etc. Or if you want to host it on your own, it's not that difficult just make sure you use something other than Apache as said above it's not really the best for static content serving. You can use other software to help like for instance if you have a large data set where people need to filter out results, you can use something like SphinxSearch. It's not just made for Search, you can use it to help find data by adding filters on it's huge index without even specifying a search query. Lets say I want to find all the users from the ages 18-24, I want them to be male, and they need to be in the zipcode xxxxx, it will return the results instantly.
I think those are all options. I think many of them should be whole separate discussions. Perhaps we should do a series? And it might be interesting to have some discussion on how memcache might fit into these architectures.
William wrote:Anyways I wrote way too much. Hopefully the posts helps someone. Like someone stated above highscalability.com is a good resource. Cal Henderson wrote a great book called "Building Scalable Web Sites". He basically built Flickr. Although it's mainly PHP/MySQL based the solutions can be used on other languages / databases with a bit of common sense.
Yes, pytrin has given a lot of helpful links above. The Cal Henderson book is 3-4 years old at this point. It is still current for people to buy? Are there newer, better books?
(#10850)
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Scalability

Post by kaisellgren »

arborint wrote:What sort of application support is needed for master/slave architectures?
Basically, all read queries (SELECT is the only one - you could automatically detect if the query is SELECT) must go to a random slave server. Meanwhile, all writes must go to a master server. This is how to support Master-Slave in its simplest form. There are other tiny things that you may want to think about; e.g., dealing with lag.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Scalability

Post by Christopher »

kaisellgren wrote:
arborint wrote:What sort of application support is needed for master/slave architectures?
Basically, all read queries (SELECT is the only one - you could automatically detect if the query is SELECT) must go to a random slave server. Meanwhile, all writes must go to a master server. This is how to support Master-Slave in its simplest form. There are other tiny things that you may want to think about; e.g., dealing with lag.
So the most basic functionality is to have separate read and write connections -- with SELECTs going to the read connection and INSERT/UPDATE/DELETE/etc. going to the write connection. I have seen that that some libraries do allow an set of read connection information to be supplied and it picks one at random. Does that end up being the simplest system? Are there other schemes to select the read connection? Any examples of implementations that people think are really good? I see a lot of references to various PHP CMS/Blog projects' solutions.

With master/master architecture would you also want to provede a set of write connection information and randomly pick one of those too?

And what are schemes to deal with lag?
(#10850)
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Scalability

Post by kaisellgren »

I just finished some sort of Master-Slave replication support for a database class I was working on. It will automatically detect SELECT by taking first 6 bytes of the prepared query and I have made clear in the documentation that you may not type anything before the SELECT keyword; e.g., you can't type comments prior to SELECT because it would ruin the check. I also provide a flag that you can set to make a SELECT query to go to the master in case you want to get data that is up-to-date because you see, the slaves may be out-dated for a minute or several (called lag). I select a slave server randomly per thread by default (can be customized if needed, e.g. random per query, but I think it makes more sense to choose randomly per thread). The process of automatically detecting the query type (read/write) makes it easier for other developers to code as they don't need to understand anything about replication which is why I'm doing it this way. If it's just you and/or a couple of others working on the project you may want to manually specify the query type. Common approaches are 2nd parameter that defines the type (master / slave query) or two query methods: rquery(), wquery() or something similar.

In Master-Slave replication, the master is for writes, and the slaves are for reads. The lag comes from the fact that when you write (to the master), if you write plenty of data or it affects lots of rows or you just have a high load in general on your SQL servers, then the binary log that the master creates will take some time to send to the slaves and get them updated. If you have several slaves (imagine e.g. 20), sending a big binary log to each of them might make the last slaves out-dated for a while. This happens in Wikipedia. Sometimes when I create an article and press the submit button, then it tells me the article doesn't exist. This is the reason, the lag causes it, the slave that was being used for me was out-dated and I got that "article does not exist, would you like to create it" even though it was indeed created. Then I just press reload after a minute and voilà the article is found. :P

With Master-Master, that is officially unsupported by MySQL as it has its own issues. Basically the commands on one master is done in another. So a failure in another server is reflected to another. Not great for backup solutions. Both masters act as a master and a slave at the same time. They both write and read. Basically, you first need to setup Master-Slave, and then edit the slave server to create binary logs (and replicate them). Then you would make the master to act also as a slave. Now, the web application would just need to select a random server to connect to for each thread and the load on your site would be pretty much balanced fifty-fifty. Simple. You must realize though that if something happens to the other server (e.g. tables were dropped), the same thing might happen to the other... depending on how did this happen (e.g. a crash and db getting corrupted wouldn't count). So, as you increase the count of servers, you increase the risk of data loss.. but I'm no expert in this area, there may be solutions to avoid these kinds of situations.

I've been analyzing mt_rand(), which is based on a matrix linear recurrence, generally generating about 7.98 bits of entropy and has a nice pseudo random distribution of bits in a statistical stance, which is just what you would look for when selecting the master server in Master-Master replication (or selecting a slave in Master-Slave). So, mt_rand(0,n) where n = count of masters (Master-Master) or slaves (Master-Slave) would be fine.
User avatar
Eran
DevNet Master
Posts: 3549
Joined: Fri Jan 18, 2008 12:36 am
Location: Israel, ME

Re: Scalability

Post by Eran »

the following article shows some useful ways to handle slave lag - http://www.mysqlperformanceblog.com/200 ... plication/
Those guys also released a mysql-master-master manager (or MMM for short :P). Seems like a useful tool - http://www.mysqlperformanceblog.com/200 ... -released/
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Scalability

Post by kaisellgren »

Cool, thanks for the MMM link.
User avatar
William
Forum Contributor
Posts: 332
Joined: Sat Oct 25, 2003 4:03 am
Location: New York City

Re: Scalability

Post by William »

arborint wrote:I want to thank you William, this is exactly the kind of practical information I was looking for.
No problem. :) If you only knew how much this forum has helped me, if you look at my original posts back around 2003 (I think) I made such a fool of myself hehe
arborint wrote:So you think Master/Master is the next logical step?
I need to re-read the discussion when I have more time (today is the 4th and have been really busy, just checked the security forums to see what advice Kai is telling people. I find it interesting. :) (thanks Kai). Then I noticed a scalability topic and was instantly interested. But to better answer your question, Master/Master is not really to handle more load, it's to make sure if one breaks you still can operate at full capacity. Servers will break eventually, and you can't afford your database server to crash out of all things. This gives you time to power up another server, have it get up to "speed" on all the latest data and have it set back into motion.
arborint wrote:What do you think about active/passive masters? It is sort of a minimal version of #1 with one read/write server and one read server.
Like I said above, Master/Master is for availability, not really scalability. What you're talking about is having a master and a slave. Masters can still always be used for reads, it's really about how much load you're getting. You can have 1% writes and 99% writes and still have a master server which is used to write to all of the other servers. The whole point of a "master" is to take writes and distribute them to slave(s). So to answer your question, thats fine but if your master dies, your slave needs to be able to become a "new" master and neither servers can ever be past 50% capacity since if one fails the other has to take everything.

What sort of application support is needed for master/slave architectures? I know that many projects have implemented database code like WordPress's HyperDB to support these architectures.
arborint wrote:And, what sort of information is useful in setting up and managing master and slave server, and bring failed server back online?
I'll have to get into this at another time.
arborint wrote:I am not sure this stuff is too practical to most programmers here, but it is interesting. It might be interesting to know which if these techniques are useful for smaller solutions.
I agree, I actually try to denormalize a bit on all my database structures. Especially for things like database counts like how many posts someone has, or comments, etc. Although thats probably considered normal practice now days, it's still not really "normalized".
arborint wrote:I think those are all options. I think many of them should be whole separate discussions. Perhaps we should do a series? And it might be interesting to have some discussion on how memcache might fit into these architectures.
Yeah that would work. Memcache is HUGE in my opinion. It can seriously reduce your server loads by like 100% or even way more. I mean think about it simply like this. You have a main web page portal that gets 100 hits per second. Now there is 3 query's on that page to get the latest comments, latest discussions, and latest news. Well thats 300 QPS. Well you can Memcache that for lets say 10 minutes. That means that you're going to go from 60000 query's in 10 minutes to 3 in 10 minutes (if done right). Kind of a bad example since if you were getting that much traffic you would have had to already done most of this. But you get the point. APC is also amazing but I'm sure most people here know that. Scalability is about being able to scale for traffic, but sometimes you need to optimize what you have before you start scaling out to more machines.
arborint wrote:Yes, pytrin has given a lot of helpful links above. The Cal Henderson book is 3-4 years old at this point. It is still current for people to buy? Are there newer, better books?
I highly recommend his book. The guy is extremely smart with this kind of thing. It goes over so much, down to things like internationalization, database scaling (sharding, replication, etc), stuff like version control for teams, to bug tracking software, to supporting stuff like UTF8, and the list just goes on and on. Plus he gives tons of examples of what Flickr has done which is great also.

Also, there has been discussion about scaling servers in terms of vertical vs horizontal on servers with expensive software. I think it is PlentyOfFish.com that actually spent over 100k on their database server. They found that the cost for performance based off software costs made it cheaper to do that then scale horizontally. Just so people know. I prefer going the open source route. Linux, Apache, MySQL, PHP, APC, Memcache, SphinxSearch, blah blah blah. But just for the few users that do work on Windows software, etc.
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Scalability

Post by kaisellgren »

I would be interested in a series that discusses different scaling approaches (Memcached, Database replication, etc).

One thing that I have noticed in scaling up vs scaling out is that scaling out becomes much more pricier if you do not use open source software as you need licenses for each machine.

As William said, Memcached can greatly reduce the load on server(s). Memcached is generally used for sessions and caching (which makes sense). One note though, Memcached is only useful if you really have high load website because otherwise you would just kill performance. For instance, if you have a small to medium site that is capable of rendering the page in 0.1 seconds, then I doubt Memcached is going to help here - not enough load. Memcached needs to operate using Internet Protocol Suite, which is pricier than having a site rendered in 0.1 seconds. It would be just better to cache and store sessions in the RAM of the same server where the site runs on using APC as long as you want to have a high performing site, but if you want to scale for other reasons then obviously APC is not going to help. For instance, if the sessions take up too much RAM and one server can't handle it, or you know that you soon need to scale anyway, then using Memcached over APC makes sense. A good rule of thumb is that the higher your site load is the more helpful Memcached is going to be. As a programmer you may have no idea how much load the site, which uses your application, might get, so, making Memcached as an on/off option sounds good.
User avatar
William
Forum Contributor
Posts: 332
Joined: Sat Oct 25, 2003 4:03 am
Location: New York City

Re: Scalability

Post by William »

I like the idea of the whole "series" thing. Wasn't DevNetwork working on a book along time ago? Anyways, maybe it would be cool to fill the Tutorial section with a series that goes over more advanced topics in PHP and web development. Things to cover stuff like:

Scalability
Availability
Security
UnitTesting
Performance
Configurations
Caching
etc

I'm not saying it should be done right away since most people with the experience to write this are are most likely very busy. There are very few books out there that cover a lot of big topics that you constantly say "wow, I didn't know that.". A small series that goes over major web development issues, with case studies, and only things that are "big" enough that most people won't know, even if they've been programming for awhile.

Any opinions? (This is kind of off topic to this discussion, but since everyone is bringing it up)
Theory?
Forum Contributor
Posts: 138
Joined: Wed Apr 11, 2007 10:43 am

Re: Scalability

Post by Theory? »

William wrote:I like the idea of the whole "series" thing. Wasn't DevNetwork working on a book along time ago? Anyways, maybe it would be cool to fill the Tutorial section with a series that goes over more advanced topics in PHP and web development. Things to cover stuff like:

Scalability
Availability
Security
UnitTesting
Performance
Configurations
Caching
etc

I'm not saying it should be done right away since most people with the experience to write this are are most likely very busy. There are very few books out there that cover a lot of big topics that you constantly say "wow, I didn't know that.". A small series that goes over major web development issues, with case studies, and only things that are "big" enough that most people won't know, even if they've been programming for awhile.

Any opinions? (This is kind of off topic to this discussion, but since everyone is bringing it up)
I would love to see a greater focus on fusion topics that deal with how PHP integrates into the grand scheme of web apps and systems. Topics like:

Programming for UX/UI
Ten reasons not to kill the Information Architect
Exposing API's
RESTful PHP

and other topics like that.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Scalability

Post by Christopher »

kaisellgren wrote:I would be interested in a series that discusses different scaling approaches (Memcached, Database replication, etc).
William wrote:I like the idea of the whole "series" thing. Wasn't DevNetwork working on a book along time ago? Anyways, maybe it would be cool to fill the Tutorial section with a series that goes over more advanced topics in PHP and web development. Things to cover stuff like:

Scalability
Availability
Security
UnitTesting
Performance
Configurations
Caching
etc
Theory? wrote:I would love to see a greater focus on fusion topics that deal with how PHP integrates into the grand scheme of web apps and systems. Topics like:

Programming for UX/UI
Ten reasons not to kill the Information Architect
Exposing API's
RESTful PHP
I think this is a very good idea and I would like to find a way to support this any way I can. I have found this very informative. It is difficult to find practical information on these subjects from the perspective of web developers who find themselves as part time system administrators -- which I think describes a large number of Devnet members.

Let me propose a process. Perhaps two or three of you could lead discussions one topic at a time. I have a high regard for the three of you and think the result would really positive. One of you would create the initial post and would be responsible for editing that first post to gather the ideas in the discussion into an article. When you feel that the discussion has created an article/tutorial that covers the topic, we can add it to the Tutorials section. Moderators could support you by removing off-topic posts at you PM request to keep things focused (we would state that off-topic posts will be removed in the first post). I'd be glad to help you in the process with any support you need. This post is just to start the discussion, so please propose alternatives or improvements to my suggestion.
(#10850)
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Scalability

Post by kaisellgren »

arborint wrote:When you feel that the discussion has created an article/tutorial that covers the topic, we can add it to the Tutorials section.
Does the move of the post to the Tutorials section mean that it will no longer be modified? I think it would be good if the article/tutorial gets updated whenever necessary (e.g. new solutions to scalability available, our experience increases as the time goes on and we know more, etc). It would be more of a "wiki".

If everyone throws in their own knowledge, I think the articles can get quite quality. Most of the current web applications are not aware of scalability or availability, so, maybe it's time to put our knowledge together and enlighten everyone.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Scalability

Post by Christopher »

I agree it needs to be editable however it is done. Discussion threads like this are good to get lots of input on the subject. I do think that having a group responsible is important because without some editorial work the information is not very valuable.
(#10850)
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Scalability

Post by kaisellgren »

Yeah, I think so. It would get messy. Anyway, I'm happy to discuss and share my knowledge, but I don't think I'm going to be an editorial, so, I'll leave that to you or someone else. ;)
Post Reply