Page 1 of 1

What hardware component is most important for this?

Posted: Mon Dec 31, 2007 5:00 pm
by seodevhead
Hey guys... I'm launching a new website that is very video intensive (HD video to be exact... much like Apple's Quicktime HD Trailers). I'm getting a new dedicated server, but because this is a new site, it's hard for me to guess what my approximate load/traffic will be, but I'm guessing about 5k-7k uniques per day each watching about 2-3 200mb videos every day.

For a website like this, I'm a little unsure what hardware I should place emphasis on when I build the server. I have the bandwidth taken care of, but between...

1) Processor
2) RAM
3) Hard drive(s)

I'm unsure which of these components is more important than the others in terms of delivering these video files to visitors the quickest. Besides just the videos, it's a standard website with php pages, mysql databases and forums and whatnot, so it's not just a video repository. Anyone have any suggestions or advice?

Posted: Mon Dec 31, 2007 5:14 pm
by Kieran Huggins
Tave you considered amazon s3? For anything on that scale you'd likely need a cluster anyway, and theirs is extremely well broken in.

Posted: Mon Dec 31, 2007 5:49 pm
by califdon
Kieran Huggins wrote:Tave you considered amazon s3? For anything on that scale you'd likely need a cluster anyway, and theirs is extremely well broken in.
I think you're referriong to their EC2. S3 is storage. Good suggestion, anyway.

Posted: Mon Dec 31, 2007 5:58 pm
by Kieran Huggins
I actually did mean S3 (for serving up all that video data).

EC2 seems poorly suited to any app that involves a database since it's stateless - I'd use either a VPS or dedicated server for the DB / web and S3 for the storage / bandwidth intensive components.

There's a calculator at Amazon for you to spec out how much it would cost, but I have a feeling it would be a more effective solution than laying out for the massive infrastructure needed.

Apple uses Akamai for their video hosting, IIRC, so it's not a radical architecture by any means.

Posted: Mon Dec 31, 2007 6:08 pm
by seodevhead
I think this Amazon EC2 and S3 is beyond the scope of what I am looking for. Besides, from my initial readings on both of these, it is WAYY beyond my skill level to manage something like that in conjunction with my server. Plus, it would put a big kink in the way scripts have been written for this new website.

Posted: Mon Dec 31, 2007 6:21 pm
by Kieran Huggins
If you estimate serving 3 TB of video traffic a day, I would strongly encourage you to think about altering a little code to accommodate it. A data model would take the pain out of saving and serving up proper URLs.

An ounce of prevention and all that.

Posted: Tue Jan 01, 2008 6:16 pm
by califdon
I support Kieran's comments. Amazon Web Services (AWS) is quite an economical proposition, too, since you can utilize really massive capacity while paying only for what you actively use. I attended a local AWS MeetUp, but haven't used the services, myself. Anyone interested in learning more about these services should check out http://aws.amazon.com and http://awsome.ws (the MeetUp group).

If you're looking for solutions, do give thought to all the potential ones available. Learning something new isn't all that hard. Look how old I am, and I'm still learning!!

@Kieran: OK on S3, but I was thinking of the server capacity, for which EC2 sounds like it could offer a flexibility and expansion capacity you could never match with your own hardware--to me, anyway. :D

Posted: Tue Jan 01, 2008 7:03 pm
by Kieran Huggins
EC2 would be AMAZING if there were some sort of local, non-volatile DB support. Right now it's next to useless for 95% of web apps :-(

Are you paying attention, Amazon? You have an opportunity to rock the web here.

Posted: Tue Jan 01, 2008 8:14 pm
by Chris Corbyn
Kieran Huggins wrote:EC2 would be AMAZING if there were some sort of local, non-volatile DB support. Right now it's next to useless for 95% of web apps :-(

Are you paying attention, Amazon? You have an opportunity to rock the web here.
What's wrong with MySQL replication? :) We manage just fine with EC2.

Posted: Tue Jan 01, 2008 10:33 pm
by califdon
Great info, Chris. As I said, I know only what I learned in one presentation by the Amazon "evangelist", but clearly there's some way to maintain state. Where is your data stored, Chris, in S3, or some other store?

Posted: Tue Jan 01, 2008 10:51 pm
by Kieran Huggins
Chris Corbyn wrote:What's wrong with MySQL replication? :) We manage just fine with EC2.
So you use an external VPS with a master DB, and each EC2 has a slave DB that replicates the entire DB when started? I guess that would work, but it still means having your own external master.

Still feels kinda hackish, in a totally non-judgmental sort of way :wink: