Page 1 of 1

PHP cURL ftp resume

Posted: Tue May 28, 2013 7:41 pm
by Eric!
Has anyone tried to get curl to resume an ftp upload? I sometimes get timeouts on large files and rather than retry, I would like it to resume.

I was thinking of checking (via curl_getinfo) that the following is true:
CURLINFO_HEADER_SIZE + CURLINFO_CONTENT_LENGTH_UPLOAD = CURLINFO_SIZE_UPLOAD

and if it does to set (via curl_setopt) CURLOPT_RESUME_FROM=CURLINFO_CONTENT_LENGTH_UPLOAD. If they are not equal, then I would restart the transfer as opposed to resuming. Does this seem like a good check?

Since I only do this transfer once in a while for backups, I thought I would run the idea by people here before experimenting with it.

Thanks.

Re: PHP cURL ftp resume

Posted: Thu May 30, 2013 2:16 pm
by mecha_godzilla
Hi,

If in doubt, check to see what commands your FTP client uses in the same situation - most FTP clients have a transcript/debug window so you can see what commands it's sending. I haven't tried using the options you mention, but there's no reason why you can't emulate the same kind of functionality with cURL.

HTH,

Mecha Godzilla

Re: PHP cURL ftp resume

Posted: Fri May 31, 2013 2:58 pm
by Eric!
Thanks for the idea. It's hard to duplicate the failure on the server to server situation. I could setup something locally to verify it works. Rather than going to that trouble I thought I'd ask, but it seems there aren't many people on here that use the curl extension as much as I do. FTP and php curl FTP extensions are different so I'm not sure how to correlate them exactly.

Re: PHP cURL ftp resume

Posted: Sat Jun 01, 2013 4:49 am
by mecha_godzilla
I'll see if I can write a test script over the weekend - I think the main issue is just replicating the conditions under which the transfer will fail. I don't think it matters whether the remote server stops responding or the connection is lost because cURL will probably still capture it as a time-out event, then you can reinitiate the connection again. You might want the script to give up after several attempts though, to stop it looping indefinitely.

M_G

Re: PHP cURL ftp resume

Posted: Mon Jun 03, 2013 6:50 pm
by mecha_godzilla
Using curl_getinfo() the output on my local machine looks something likes this:

Code: Select all

array(21) {
  ["url"]=>
  string(56) "ftp://0.0.0.0/my_file.zip"
  ["content_type"]=>
  NULL
  ["http_code"]=>
  int(350)
  ["header_size"]=>
  int(0)
  ["request_size"]=>
  int(0)
  ["filetime"]=>
  int(-1)
  ["ssl_verify_result"]=>
  int(0)
  ["redirect_count"]=>
  int(0)
  ["total_time"]=>
  float(3.064527)
  ["namelookup_time"]=>
  float(0.018974)
  ["connect_time"]=>
  float(0.358245)
  ["pretransfer_time"]=>
  float(1.986533)
  ["size_upload"]=>
  float(0)
  ["size_download"]=>
  float(0)
  ["speed_download"]=>
  float(0)
  ["speed_upload"]=>
  float(0)
  ["download_content_length"]=>
  float(314308)
  ["upload_content_length"]=>
  float(0)
  ["starttransfer_time"]=>
  float(0)
  ["redirect_time"]=>
  float(0)
  ["certinfo"]=>
  array(0) {
  }
}
Based on that output, you could just use the "download_content_length" value to compare the size of the file on the remote server with the local copy.

In my FTP client, when trying to resume a file upload the command transcript was as follows (note that the file sizes are different because these commands were run prior to resuming the upload):

Code: Select all

Cmd: TYPE I
200: TYPE is now 8-bit binary
Cmd: OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid
200:  MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
Cmd: MLST my_file.zip
250: Begin
      type=file;size=75342;modify=20130603231455;UNIX.mode=0644;UNIX.uid=2778;UNIX.gid=2779;unique=851g1609455; my_file.zip
     End.
Cmd: PORT 0,0,0,0,0,0
200: PORT command successful
Cmd: REST 75342
350: Restarting at 75342
Cmd: STOR my_file.zip
150: Connecting to port 49650
So, as long as the cURL functions work in the same way, it should be possible to resume your uploads without any problems - I've only ever had one instance where my FTP client got confused about a file's size and didn't download it correctly, so you should be safe to resume. You could call a shell script to check the backup file with MD5 or SHA1 after the transfer has been completed to be on the safe side.

M_G