Page 1 of 1
Flavours for server farms, and how-to?
Posted: Tue Jan 06, 2004 7:28 pm
by JAM
Questions...
What flavours are:
...best suited for server farms. Any big difference?
...can make use of 2-X CPU's. And does that require kernel editing?
Server farms:
...can someone guide me to more info about actually building those? I have a couple of ideas and alot of "I think that it's done like this" but I could really use more guidance in that area.
A quite oversized (yes, huge) connection to the inet might be available for me in the future, and setting up boxes with linux running game-servers would not be a problem. But I want to explore the world of actually hosting other stuff also, so I need to begin reading up as of today.
Thanks in advance.
Posted: Tue Jan 06, 2004 11:31 pm
by evilMind
Sorcerer, Slackware , and FreeBSD are my personal favorites.
Making use of 2 cpus may or may not require kernel compiling. It depends on which flavour you are running if you have a default SMP capable kernel.
However, it's always best to recompile the kernel any ways - that way you can add optimization flags to the build. (eg -march=athlon-xp or -march=k6-3) and cut out any nonsense that was built in by default, make more modules, etc..
Don't have much advice on setting up server farms at this time.
If you are looking for something to play around with (testing distributed systems) check out
http://www-itg.lbl.gov/mbone/ <- that may be of no use, but it is fun to get up and running.
Posted: Wed Jan 07, 2004 9:51 am
by Pyrite
Slackware. Debian. seems 2b good server distros.
Posted: Wed Jan 07, 2004 6:09 pm
by JAM
Thanks. Slack and *BSD was mentioned by a couple of friends discussing the differences (I didn't understand much but...) and hopefully before the times come I might have picked up some more along the way.
But is the multible cpu-support in the kernel and not the flavour? Meaning, would I be able to use virtually any flavour and get 2 or more cpus to work together by recompiling?
May sound wierd, but I would prefer 'playing around' with something trying to get it to work, when an advanced user would know at starters that it's impossible...

Posted: Wed Jan 07, 2004 6:29 pm
by Pyrite
SMP support is in the Linux kernel. And most distros have an smp-enabled precompiled stock kernel avaible.
Posted: Wed Jan 07, 2004 6:36 pm
by evilMind
** excerpt from the Symmetric Multi-Processor support "Help" option (modified a little bit) **
A kernel built w/o SMP support will run on single and multiprocessor machines, but will use only one CPU of a multiprocessor machine. Note that if you say yes to SMP Support and choose architecture "586" or "Pentium" under "Processor family", the kernel will not work on 486 architectures. Similarly, multiprocessor kernels for the "PPro" architecture may not work on all Pentium based boards. People using multiprocessor machines who say yes to SMP support should also say yes to "Enhanced Real Time Clock Support", below.
** end of excerpt **
Yes the SMP support is in the kernel. Also, SMP is built into some programs (gimp has an option to utilize 2 or more cpu's). However, It all comes back to the kernel being able to support 2 or more CPU's.
As far as the flavour, I meant that some distros come with a prebuilt SMP kernel. Some come with a kernel that does not have smp built into it. Most distro's that I've used leave a .config file in the kernel source tree. If the distro left the .config file you could check by going into the src tree, type make menuconfig, go under Processor type and features --->, and see if SMP is on. If not, you may be able to tell from the Symbols left over in the System.map file, but since I don't have SMP (only one processor), I couldn't tell you the exact sym to look for.
If you are using a 2.4.* kernel then go to
"Processor type and features --->"
and select Symmetric multi-processing support.
If you are using a 2.6.* kernel it is under the same heading but you should also set the max number of CPU's that can/could be used. Also, you may want to say yes to "Preempitible Kernel" if you are building a kernel for a desktop, embedded, or real-time system.
Hope that clears some things up.
Posted: Wed Jan 07, 2004 6:44 pm
by Pyrite
Geez dude, overkill.
Posted: Wed Jan 21, 2004 10:13 am
by BDKR
I'm going to talk primarily about your server farm question.
I built an LVS for a lottery operation in Venezuela. LVS stands for Linux Virtual Server. I also call it a cluster.
Before we go any further, there are (at least) two kinds of clusters (there may be more that I don't know about). The first is what you are looking for where there are groups (farms or pools) of machines that all handle the same tasks, BUT DO IT WITHOUT INTERACTING WITH ONE ANOTHER. With these types of clusters (google is an example of one), you can add machines to the cluster to scale the site overall. All of the machines are load balanced using something akin to a round-robin or weighted round-robin algorithm. There is normally a cluster controller (sometimes called a router or ATM standing for advance traffic manager) that routes traffic to individual machines in the cluster. It can also check for failures of hardware and software and perform certain actions (based on what you need) should it encounter a failure.
In our case, we had a pool of web servers (2), a pool of transaction servers (6), and a pool of databases (3). There are two traffic managers (one as back up). We also had a backup firewall sitting on it's own IP address. Not to mention that we also built a wireless network that stretched across a good portion of the eastern half of Margarita Island.
My backend and network chores were huge!
Anyway, the other type is a like a Beowulf cluster. More of a distributed processing cluster. All of those Super computers you hear about are of this type. Not really practical for what you need.
One more thing, almost ANY of the distros out there can do the job.
To wrap this up, we used Turbo Cluster from Turbo Linux. It was good for me as I was a Cluster first timer at the implementation level. We were about to deploy a second data center on the mainland based on Pirhana, which is the free equivalent. It comes with a lot of Red Hat releases. I don't know about 9.
You can also check out the website below.
http://www.linuxvirtualserver.org/
There is a lot more to it than what I mentioned here as there will also be a number of sticky issues forcing you to build some rather creative solutions.
If you need any help with this, or have more questions, drop me a PM.
Cheers,
BDKR
Posted: Wed Jan 21, 2004 10:21 am
by BDKR
One more thing. Keep the dual proc machines for database servers. They will REALLY NEED TO POWER. Trust me. Other servers could use it as well, but consider that the database is normally the first bottleneck in any setup. That said, focus first on it's performance.
We had two Supermicro based pedestal servers with SuperDLE twin PIII mobos. 1.5 gigs of memory each and Utlra 160 controllers. One box was running raid 2 and the other was raid 5. They were RH 6.2 running MySQL 3.23.49. They were rather tough to get setup real well for a long list of reasons that has nothing to do with Linux or MySQL, but one they were up to full song, I benched performance that peaked at just over 2100 inserts a second.
Those boxes were monsters!
We also had a third single proc machine that we used to help during restoration (or should I say disaster recovery) procedures. Do to the problems and difficulties with MySQL's replication (or anybodys for that matter), it was easier to have that third box.
yada yada yada.....
Cheers,
BDKR