Advice on Deployment
Moderator: General Moderators
Advice on Deployment
I'm currently quite troubled by the way of deployment my team is adopting... It's very old-fashioned and I know it doesn't work very well. But I don't exactly know how to change it, so please give some suggestions about it...
Here is our current setup:
2 webservers
1 database server
1 test server
Current deployment adaptation
1. We develop and work on the test server, every changes is uploaded manually to the test server.
2. When a change or feature is complete, we then commit the changes to SVN repository.
3. After committing the changes, we then upload our changes to the first webserver, where there will be a cronjob running every minute to sync the files between the servers.
Something very annoying is, whenever we upload a file just as the syncing job starts, the file that is sync-ed will appear corrupted, since it is only half-uploaded. Another thing is whenever there is a deployment fault, it will be extremely difficult to revert. These are basically the problem I'm facing, what should I do?
Here is our current setup:
2 webservers
1 database server
1 test server
Current deployment adaptation
1. We develop and work on the test server, every changes is uploaded manually to the test server.
2. When a change or feature is complete, we then commit the changes to SVN repository.
3. After committing the changes, we then upload our changes to the first webserver, where there will be a cronjob running every minute to sync the files between the servers.
Something very annoying is, whenever we upload a file just as the syncing job starts, the file that is sync-ed will appear corrupted, since it is only half-uploaded. Another thing is whenever there is a deployment fault, it will be extremely difficult to revert. These are basically the problem I'm facing, what should I do?
Re: Advice on Deployment
It actually doesn't sound like a bad system, but I'd increase the sync time to maybe every hour, and that should decrease the number of randomly corrupted files. It also allows you to better test an update, because you'll know that your patches should be stable every hour on the hour, or something like that.
Re: Advice on Deployment
Having different files on 2 different web servers for a whole hour could be a problem. I'd recommend changing your cron job to not transfer any files that were modified in the last minute. That way, files that are being uploaded won't be touched.
Actually, now that I think about it - why have a cron job at all? If you're manually uploading to the first web server, it should b easy to manually trigger the transfer to the second. This'll cut down on useless clock cycles, and will ensure both servers are up-to-date.
Actually, now that I think about it - why have a cron job at all? If you're manually uploading to the first web server, it should b easy to manually trigger the transfer to the second. This'll cut down on useless clock cycles, and will ensure both servers are up-to-date.
Real programmers don't comment their code. If it was hard to write, it should be hard to understand.
Re: Advice on Deployment
Hm, I was under the impression the second server was more of a backup, but that's probably just me misinterpreting.
Re: Advice on Deployment
Use rsync.
- Christopher
- Site Administrator
- Posts: 13596
- Joined: Wed Aug 25, 2004 7:54 pm
- Location: New York, NY, US
Re: Advice on Deployment
Seems like rather than having server2 sync from server1 that you should just push out changes to both server when you do a release. The sync once a minute sounds almost dangerous -- imagine server1 gets hacked or a file gets corrupted.
(#10850)
Re: Advice on Deployment
Files changing on the webservers are not only the source files, but some other user uploaded images and some .po translation files as well... Is it not a bad system?
- Christopher
- Site Administrator
- Posts: 13596
- Joined: Wed Aug 25, 2004 7:54 pm
- Location: New York, NY, US
Re: Advice on Deployment
It seems like you should have a separate sync system for those uploads. That is a different problem that deploying new software.TheOnly92 wrote:Files changing on the webservers are not only the source files, but some other user uploaded images and some .po translation files as well... Is it not a bad system?
(#10850)
Re: Advice on Deployment
Disclaimer: blatant self-promotion ahead, but I think it's also right on topic ...
I just wrote a book on PHP tools. One of the chapters deals with exactly the problem you are facing as well: "Deploying Applications."
I think that application deployment should meet the following two requirements:
1. It should be fast (preferably automated) so as not to inconvenience users.
2. It should be completely reversible (in case something goes wrong).
One way to solve this problem is to script the deployment process. Completely automating the process speeds it up and eliminates human error. The tool I use is Phing (http://phing.info/trac/), which is a project build system. Specifically, I created a Phing build file to:
- check out the files from the source code repository
- pre-process files, such as config files
- take down the production site and display a maintenance message
- backup the DB
- install the files
- upgrade the DB
- rotate log files
- take down the maintenance page
- restart the web server
Each of the steps is completely reversible. Moreover, since everything is scripted, deployment happens within a few seconds. The way I set it up, it supports any number of environment (development, test, staging, production, etc.) via simple configuration files.
If you're interested in reading more about how to automate application deployment in PHP, you might want to check out my book, "Expert PHP 5 Tools":
http://www.packtpub.com/expert-php-5-tools/book
I just wrote a book on PHP tools. One of the chapters deals with exactly the problem you are facing as well: "Deploying Applications."
I think that application deployment should meet the following two requirements:
1. It should be fast (preferably automated) so as not to inconvenience users.
2. It should be completely reversible (in case something goes wrong).
One way to solve this problem is to script the deployment process. Completely automating the process speeds it up and eliminates human error. The tool I use is Phing (http://phing.info/trac/), which is a project build system. Specifically, I created a Phing build file to:
- check out the files from the source code repository
- pre-process files, such as config files
- take down the production site and display a maintenance message
- backup the DB
- install the files
- upgrade the DB
- rotate log files
- take down the maintenance page
- restart the web server
Each of the steps is completely reversible. Moreover, since everything is scripted, deployment happens within a few seconds. The way I set it up, it supports any number of environment (development, test, staging, production, etc.) via simple configuration files.
If you're interested in reading more about how to automate application deployment in PHP, you might want to check out my book, "Expert PHP 5 Tools":
http://www.packtpub.com/expert-php-5-tools/book
Re: Advice on Deployment
Thank you for joining and contributing. I picked up a copy. The E-Book download is empty so I contacted support. Feel free to stop by in your spare time and knock off a few questions 
Re: Advice on Deployment
It seems to be a nice book, will try to buy it someday 
I have also thought about using Phing as a deployment tool, but I wasn't sure where to put Phing, as my webservers are protected by vlan and inaccessible directly, whereas the test server and the SVN server is located in another network, so it seems impossible to access my webservers from the test server or vice versa. In this case, how should I set up the Phing deployment script?
I have also thought about using Phing as a deployment tool, but I wasn't sure where to put Phing, as my webservers are protected by vlan and inaccessible directly, whereas the test server and the SVN server is located in another network, so it seems impossible to access my webservers from the test server or vice versa. In this case, how should I set up the Phing deployment script?
Re: Advice on Deployment
The fact that all servers are in difference locations and networks shouldn't really be an issue. Phing lets you check out SVN projects remotely. Also, I'm assuming you can access the web server remotely via SSH. In that case, you can leverage Phing's support for SSH and SCP commands to do most of the work remotely while executing the actual build script on your local machine.
Another option is to install Phing on the server, copy the build script over, and execute it on the server.
Another option is to install Phing on the server, copy the build script over, and execute it on the server.