Starter Web Availability Fault Tolerance

Whether you are using Linux on the desktop or as a server, it's still good that you're using Linux. Linux related questions go here.

Moderator: General Moderators

Post Reply
User avatar
volomike
Forum Regular
Posts: 633
Joined: Wed Jan 16, 2008 9:04 am
Location: Myrtle Beach, South Carolina, USA

Starter Web Availability Fault Tolerance

Post by volomike »

I'm now working for a startup out of Boston and we're in discussions about server availability so that we can sustain outages for like 4 hours, and also handle higher web traffic. What is a starter web availability system that you would recommend, and is there anything "canned" out there that makes this easier to setup and test? We want to get this going separately like a lab, and then when it all works, we start doing limited cutover testing to see if we can move to this.

Currently we spend around $2500 per day in advertising, and have a $20 transaction coming across our single server every minute to 3 minutes, sometimes even triple that.

Note -- I love cPanel, and have pretty decent Linux skills, but will probably need extra help on some of the seriously tough Linux config stuff, like mail, DNS, fending off DDOS, some of the more advanced firewall stuff, etc.

I've thought about doing 3 Linode servers installed with centOS/cPanel. So we'd have 2 web nodes and one MySQL database server. The 2 web nodes would need a fast replication system so that I could change a file on one and it would be on the second web node within a very short period. So, on fast replication, I don't think NFS will be fast enough -- right? I mean, NFS might have like a delay of several hours, right? So, if we have a 4 hour maintenance on one web node, we can still limp along. But on the database server, we'd have to implement a 4th server (a dev server used as standby) and then cutover to it when our central database server is being worked on.

Okay, the above is all well and good, but the trouble I'll have are some of the heavy duty Linux tasks such as fending off DDOS, anything more than simple firewall rules (I'm pretty good with iptables, but not good enough to fend off DDOS), advanced DNS, advanced mail config, etc.

P.S. My background is that I've had a taste of some of this while working at ADP. There, we used 1U servers (and eventually, blade servers) and built all this using Suse Linux. (I'm not a fan of Suse. I prefer centOS + cPanel. I would probably prefer Ubuntu Server, but Webmin is so dang hard to use with it for doing mail configs, DNS tasks, setting up FTP, etc.)
User avatar
Christopher
Site Administrator
Posts: 13592
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Starter Web Availability Fault Tolerance

Post by Christopher »

volomike wrote:I've thought about doing 3 Linode servers installed with centOS/cPanel. So we'd have 2 web nodes and one MySQL database server. The 2 web nodes would need a fast replication system so that I could change a file on one and it would be on the second web node within a very short period. So, on fast replication,
You may want to look into a cluster backend and multiple frontend servers behind a load balancer.
volomike wrote:I don't think NFS will be fast enough -- right? I mean, NFS might have like a delay of several hours, right?
NFS is a file system so should be very fast. The problem is when NFS hangs, but that should not be a problem in your case.

Okay, the above is all well and good, but the trouble I'll have are some of the heavy duty Linux tasks such as fending off DDOS, anything more than simple firewall rules (I'm pretty good with iptables, but not good enough to fend off DDOS), advanced DNS, advanced mail config, etc.

P.S. My background is that I've had a taste of some of this while working at ADP. There, we used 1U servers (and eventually, blade servers) and built all this using Suse Linux. (I'm not a fan of Suse. I prefer centOS + cPanel. I would probably prefer Ubuntu Server, but Webmin is so dang hard to use with it for doing mail configs, DNS tasks, setting up FTP, etc.)
(#10850)
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Starter Web Availability Fault Tolerance

Post by VladSun »

There are 10 types of people in this world, those who understand binary and those who don't
Doug G
Forum Contributor
Posts: 282
Joined: Sun Sep 09, 2007 6:27 pm

Re: Starter Web Availability Fault Tolerance

Post by Doug G »

If you like open source, ispconfig3 supports clustered servers.
User avatar
volomike
Forum Regular
Posts: 633
Joined: Wed Jan 16, 2008 9:04 am
Location: Myrtle Beach, South Carolina, USA

Re: Starter Web Availability Fault Tolerance

Post by volomike »

I called RackSpace and got a quote of around $4000 to $5000 per month for a 2 web node, one MySQL server, shared NFS volume solution. So, that's a bit too rich for us right now. We love RackSpace tech support, uptime stats, security, and alerts -- so we will stick with them. However, eventually we do plan to go with this $4000 to $5000 per month solution. We're hoping our transactions are high enough soon that we can afford it.
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: Starter Web Availability Fault Tolerance

Post by josh »

I use http://linode.com you can deploy servers with their API, or by clicking a button to clone an existing server. You can deploy load balancers the same way w/ automatic failover. From there you just choose any server as the master for web (and/or DB), and sync to all the slaves (using rsync, and mysql replication). You could have a fully redundant cluster for under $100.

Also if you need true dedicated, my friend users this which is way cheaper than rackspace - https://hivelocity.net/dedicated-servers/specials/

I know people say rackspace is worth it, and for some people it is... but I work with both rackspace & linode daily and linode responds within seconds whereas rackspace sometimes doesn't get back to us until the next day, and both are equally competent (the difference is rackspace will go in & edit your configuration for you, linode will just link you to an article on how to do it - but I prefer that anyways.)

With any company like rackspace or linode your single point of failure is the web host. You'd need to choose multiple companies & be prepared to failover to the other web host, by editing the DNS entries. Cloudflare/Amazon does this automatically - but then your single point of failure is cloudflare/Amazon - I work with cloudflare all the time, they go down a lot - amazon went down a month ago and lost $100 million in 30 minutes. You can have multiple DNS servers with different DNS providers, or host your own DNS at multiple data centers, and then edit your DNS at whichever DNS is still online after a failure. The downside is you need tech staff on standby to edit the DNS 24/7, or write your own custom scripts - either one can fail.

Of course all of this tech could still fail from human error, so you need good monitoring - not just ping monitoring but something that pokes around your website & makes sure coding errors aren't preventing checkouts. Maybe some custom alerts that go out if you go X amount of minutes without making a sale. Linode also has some alerts for when CPU averages 90% over 2hrs, and things like disk space running low which helps to catch issues before they are issues.

Recently the ".io" TLD's root nameservers went down. Anyone with an ".io" domain name was offline, and there was nothing they could do about it. Aside from maybe having an alternate domain that is publicized & emailed out to your users during an outage at that point.
Post Reply