Page 1 of 1

Server Solution (?) For File Storage/Archive Site

Posted: Thu Oct 16, 2014 12:25 pm
by Skara
I am potentially building a site for users to archive gigabytes of data at a time.

The way this would work is having a drag-and-drop space for complete archives that need to go into cold storage. These might range from a few gigabytes to half a terabyte in size. (We would also offer a mail-your-drive-to-us option.) After uploading the data, we would have a small local server poll the site, pull the data off, and put it on our cold storage system. Once properly archived and backed up, the system would remove the files from the website server, leaving only entries for the user that they had been archived. (To un-archive the data, we would have to pull it and upload or mail it to them.) So... If you're familiar with it, similar to Amazon Glacier. But on a smaller, more focused scale with a very specific market in mind.

My question is about what kind of server/hosting I should develop this on. Would someone like Bluehost have a solution available, or should I look into something like Heroku? What are my options here? And what (non-obvious) bottlenecks should I watch out for? To start this project off, I aim to develop in PHP/MySQL and, if a successful project, possibly later migrate to something else.

I would appreciate any thoughts or advice anyone has.

Thanks!

Re: Server Solution (?) For File Storage/Archive Site

Posted: Fri Oct 17, 2014 3:56 pm
by Christopher
Heroku or Cassandra (http://cassandra.apache.org/) would be the place to start. You probably want to find/write an upload manager to deal with large uploads. You also might look into various queue system or workers (e.g., Gearman) to offload time consuming processes like generating indexes, compressing archives or emailing to other servers.

Re: Server Solution (?) For File Storage/Archive Site

Posted: Thu Oct 23, 2014 9:56 am
by Skara
Thanks. Cassandar and Gearman look interesting.