Linux command to transfer only the updated parts of files.
Moderator: General Moderators
-
impulse()
- Forum Regular
- Posts: 748
- Joined: Wed Aug 09, 2006 8:36 am
- Location: Staffordshire, UK
- Contact:
Linux command to transfer only the updated parts of files.
I was reading a few months back of a Linux command that sends a file to a server and if the file exists already it only sends the parts of the file that need updating to bring it to it's latest state.
Could somebody point me in the right direction please?
Could somebody point me in the right direction please?
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
-
impulse()
- Forum Regular
- Posts: 748
- Joined: Wed Aug 09, 2006 8:36 am
- Location: Staffordshire, UK
- Contact:
I've read the documentation that says RSYNC compresses the data before it's sent but what's happening at the moment is silly.
I've deleted the files that is creates as backups so RSYNC transfers the full file and it seems to be transferring a 14MB from a 144kbps UP connection in minutes. This used to take about 10-15 minutes with SCP but now it seems to do it in about 3-5 minutes.
I can confirm it's transferring the full 14MB file and not only missing bytes. The backup seems to untar OK when it's received so I can't see any problems with it.
Is this normal behavior for RSYNC?
I've deleted the files that is creates as backups so RSYNC transfers the full file and it seems to be transferring a 14MB from a 144kbps UP connection in minutes. This used to take about 10-15 minutes with SCP but now it seems to do it in about 3-5 minutes.
I can confirm it's transferring the full 14MB file and not only missing bytes. The backup seems to untar OK when it's received so I can't see any problems with it.
Is this normal behavior for RSYNC?
- Ollie Saunders
- DevNet Master
- Posts: 3179
- Joined: Tue May 24, 2005 6:01 pm
- Location: UK
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
-
impulse()
- Forum Regular
- Posts: 748
- Joined: Wed Aug 09, 2006 8:36 am
- Location: Staffordshire, UK
- Contact:
I thought Subversion would include some manual work.
I've setup SSH keys and dropped Rsync in a CRON job so it's completely automated, which is what I need.
I've even wrote a nifty little PHP script which will create a folder every Sunday night named 110707-180707 (or whatever the date is for 7 days before) and drop all current contents into there. Then start putting backups into the main directory again. I felt very smug when it all came together and worked
I've setup SSH keys and dropped Rsync in a CRON job so it's completely automated, which is what I need.
I've even wrote a nifty little PHP script which will create a folder every Sunday night named 110707-180707 (or whatever the date is for 7 days before) and drop all current contents into there. Then start putting backups into the main directory again. I felt very smug when it all came together and worked
- Ollie Saunders
- DevNet Master
- Posts: 3179
- Joined: Tue May 24, 2005 6:01 pm
- Location: UK
But isn't that a feature of rsync?feyd wrote:Typically if the files are different, the newer file will override. Meaning the files aren't compared to see what's different, that would waste a lot of time.
http://samba.anu.edu.au/rsync/features.html wrote:rsync uses the "rsync algorithm" which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.