Server Solution (?) For File Storage/Archive Site

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

Post Reply
User avatar
Skara
Forum Regular
Posts: 703
Joined: Sat Mar 12, 2005 7:13 pm
Location: US

Server Solution (?) For File Storage/Archive Site

Post by Skara »

I am potentially building a site for users to archive gigabytes of data at a time.

The way this would work is having a drag-and-drop space for complete archives that need to go into cold storage. These might range from a few gigabytes to half a terabyte in size. (We would also offer a mail-your-drive-to-us option.) After uploading the data, we would have a small local server poll the site, pull the data off, and put it on our cold storage system. Once properly archived and backed up, the system would remove the files from the website server, leaving only entries for the user that they had been archived. (To un-archive the data, we would have to pull it and upload or mail it to them.) So... If you're familiar with it, similar to Amazon Glacier. But on a smaller, more focused scale with a very specific market in mind.

My question is about what kind of server/hosting I should develop this on. Would someone like Bluehost have a solution available, or should I look into something like Heroku? What are my options here? And what (non-obvious) bottlenecks should I watch out for? To start this project off, I aim to develop in PHP/MySQL and, if a successful project, possibly later migrate to something else.

I would appreciate any thoughts or advice anyone has.

Thanks!
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Server Solution (?) For File Storage/Archive Site

Post by Christopher »

Heroku or Cassandra (http://cassandra.apache.org/) would be the place to start. You probably want to find/write an upload manager to deal with large uploads. You also might look into various queue system or workers (e.g., Gearman) to offload time consuming processes like generating indexes, compressing archives or emailing to other servers.
(#10850)
User avatar
Skara
Forum Regular
Posts: 703
Joined: Sat Mar 12, 2005 7:13 pm
Location: US

Re: Server Solution (?) For File Storage/Archive Site

Post by Skara »

Thanks. Cassandar and Gearman look interesting.
Post Reply