Has anyone tried to get curl to resume an ftp upload? I sometimes get timeouts on large files and rather than retry, I would like it to resume.
I was thinking of checking (via curl_getinfo) that the following is true:
CURLINFO_HEADER_SIZE + CURLINFO_CONTENT_LENGTH_UPLOAD = CURLINFO_SIZE_UPLOAD
and if it does to set (via curl_setopt) CURLOPT_RESUME_FROM=CURLINFO_CONTENT_LENGTH_UPLOAD. If they are not equal, then I would restart the transfer as opposed to resuming. Does this seem like a good check?
Since I only do this transfer once in a while for backups, I thought I would run the idea by people here before experimenting with it.
Thanks.
PHP cURL ftp resume
Moderator: General Moderators
- mecha_godzilla
- Forum Contributor
- Posts: 375
- Joined: Wed Apr 14, 2010 4:45 pm
- Location: UK
Re: PHP cURL ftp resume
Hi,
If in doubt, check to see what commands your FTP client uses in the same situation - most FTP clients have a transcript/debug window so you can see what commands it's sending. I haven't tried using the options you mention, but there's no reason why you can't emulate the same kind of functionality with cURL.
HTH,
Mecha Godzilla
If in doubt, check to see what commands your FTP client uses in the same situation - most FTP clients have a transcript/debug window so you can see what commands it's sending. I haven't tried using the options you mention, but there's no reason why you can't emulate the same kind of functionality with cURL.
HTH,
Mecha Godzilla
Re: PHP cURL ftp resume
Thanks for the idea. It's hard to duplicate the failure on the server to server situation. I could setup something locally to verify it works. Rather than going to that trouble I thought I'd ask, but it seems there aren't many people on here that use the curl extension as much as I do. FTP and php curl FTP extensions are different so I'm not sure how to correlate them exactly.
- mecha_godzilla
- Forum Contributor
- Posts: 375
- Joined: Wed Apr 14, 2010 4:45 pm
- Location: UK
Re: PHP cURL ftp resume
I'll see if I can write a test script over the weekend - I think the main issue is just replicating the conditions under which the transfer will fail. I don't think it matters whether the remote server stops responding or the connection is lost because cURL will probably still capture it as a time-out event, then you can reinitiate the connection again. You might want the script to give up after several attempts though, to stop it looping indefinitely.
M_G
M_G
- mecha_godzilla
- Forum Contributor
- Posts: 375
- Joined: Wed Apr 14, 2010 4:45 pm
- Location: UK
Re: PHP cURL ftp resume
Using curl_getinfo() the output on my local machine looks something likes this:
Based on that output, you could just use the "download_content_length" value to compare the size of the file on the remote server with the local copy.
In my FTP client, when trying to resume a file upload the command transcript was as follows (note that the file sizes are different because these commands were run prior to resuming the upload):
So, as long as the cURL functions work in the same way, it should be possible to resume your uploads without any problems - I've only ever had one instance where my FTP client got confused about a file's size and didn't download it correctly, so you should be safe to resume. You could call a shell script to check the backup file with MD5 or SHA1 after the transfer has been completed to be on the safe side.
M_G
Code: Select all
array(21) {
["url"]=>
string(56) "ftp://0.0.0.0/my_file.zip"
["content_type"]=>
NULL
["http_code"]=>
int(350)
["header_size"]=>
int(0)
["request_size"]=>
int(0)
["filetime"]=>
int(-1)
["ssl_verify_result"]=>
int(0)
["redirect_count"]=>
int(0)
["total_time"]=>
float(3.064527)
["namelookup_time"]=>
float(0.018974)
["connect_time"]=>
float(0.358245)
["pretransfer_time"]=>
float(1.986533)
["size_upload"]=>
float(0)
["size_download"]=>
float(0)
["speed_download"]=>
float(0)
["speed_upload"]=>
float(0)
["download_content_length"]=>
float(314308)
["upload_content_length"]=>
float(0)
["starttransfer_time"]=>
float(0)
["redirect_time"]=>
float(0)
["certinfo"]=>
array(0) {
}
}In my FTP client, when trying to resume a file upload the command transcript was as follows (note that the file sizes are different because these commands were run prior to resuming the upload):
Code: Select all
Cmd: TYPE I
200: TYPE is now 8-bit binary
Cmd: OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid
200: MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
Cmd: MLST my_file.zip
250: Begin
type=file;size=75342;modify=20130603231455;UNIX.mode=0644;UNIX.uid=2778;UNIX.gid=2779;unique=851g1609455; my_file.zip
End.
Cmd: PORT 0,0,0,0,0,0
200: PORT command successful
Cmd: REST 75342
350: Restarting at 75342
Cmd: STOR my_file.zip
150: Connecting to port 49650M_G