Page 1 of 1
PHP cURL ftp resume
Posted: Tue May 28, 2013 7:41 pm
by Eric!
Has anyone tried to get curl to resume an ftp upload? I sometimes get timeouts on large files and rather than retry, I would like it to resume.
I was thinking of checking (via curl_getinfo) that the following is true:
CURLINFO_HEADER_SIZE + CURLINFO_CONTENT_LENGTH_UPLOAD = CURLINFO_SIZE_UPLOAD
and if it does to set (via curl_setopt) CURLOPT_RESUME_FROM=CURLINFO_CONTENT_LENGTH_UPLOAD. If they are not equal, then I would restart the transfer as opposed to resuming. Does this seem like a good check?
Since I only do this transfer once in a while for backups, I thought I would run the idea by people here before experimenting with it.
Thanks.
Re: PHP cURL ftp resume
Posted: Thu May 30, 2013 2:16 pm
by mecha_godzilla
Hi,
If in doubt, check to see what commands your FTP client uses in the same situation - most FTP clients have a transcript/debug window so you can see what commands it's sending. I haven't tried using the options you mention, but there's no reason why you can't emulate the same kind of functionality with cURL.
HTH,
Mecha Godzilla
Re: PHP cURL ftp resume
Posted: Fri May 31, 2013 2:58 pm
by Eric!
Thanks for the idea. It's hard to duplicate the failure on the server to server situation. I could setup something locally to verify it works. Rather than going to that trouble I thought I'd ask, but it seems there aren't many people on here that use the curl extension as much as I do. FTP and php curl FTP extensions are different so I'm not sure how to correlate them exactly.
Re: PHP cURL ftp resume
Posted: Sat Jun 01, 2013 4:49 am
by mecha_godzilla
I'll see if I can write a test script over the weekend - I think the main issue is just replicating the conditions under which the transfer will fail. I don't think it matters whether the remote server stops responding or the connection is lost because cURL will probably still capture it as a time-out event, then you can reinitiate the connection again. You might want the script to give up after several attempts though, to stop it looping indefinitely.
M_G
Re: PHP cURL ftp resume
Posted: Mon Jun 03, 2013 6:50 pm
by mecha_godzilla
Using curl_getinfo() the output on my local machine looks something likes this:
Code: Select all
array(21) {
["url"]=>
string(56) "ftp://0.0.0.0/my_file.zip"
["content_type"]=>
NULL
["http_code"]=>
int(350)
["header_size"]=>
int(0)
["request_size"]=>
int(0)
["filetime"]=>
int(-1)
["ssl_verify_result"]=>
int(0)
["redirect_count"]=>
int(0)
["total_time"]=>
float(3.064527)
["namelookup_time"]=>
float(0.018974)
["connect_time"]=>
float(0.358245)
["pretransfer_time"]=>
float(1.986533)
["size_upload"]=>
float(0)
["size_download"]=>
float(0)
["speed_download"]=>
float(0)
["speed_upload"]=>
float(0)
["download_content_length"]=>
float(314308)
["upload_content_length"]=>
float(0)
["starttransfer_time"]=>
float(0)
["redirect_time"]=>
float(0)
["certinfo"]=>
array(0) {
}
}
Based on that output, you could just use the "download_content_length" value to compare the size of the file on the remote server with the local copy.
In my FTP client, when trying to resume a file upload the command transcript was as follows (note that the file sizes are different because these commands were run prior to resuming the upload):
Code: Select all
Cmd: TYPE I
200: TYPE is now 8-bit binary
Cmd: OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid
200: MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
Cmd: MLST my_file.zip
250: Begin
type=file;size=75342;modify=20130603231455;UNIX.mode=0644;UNIX.uid=2778;UNIX.gid=2779;unique=851g1609455; my_file.zip
End.
Cmd: PORT 0,0,0,0,0,0
200: PORT command successful
Cmd: REST 75342
350: Restarting at 75342
Cmd: STOR my_file.zip
150: Connecting to port 49650
So, as long as the cURL functions work in the same way, it should be possible to resume your uploads without any problems - I've only ever had one instance where my FTP client got confused about a file's size and didn't download it correctly, so you should be safe to resume. You could call a shell script to check the backup file with MD5 or SHA1 after the transfer has been completed to be on the safe side.
M_G