PHP cURL ftp resume

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

PHP cURL ftp resume

Post by Eric! »

Has anyone tried to get curl to resume an ftp upload? I sometimes get timeouts on large files and rather than retry, I would like it to resume.

I was thinking of checking (via curl_getinfo) that the following is true:
CURLINFO_HEADER_SIZE + CURLINFO_CONTENT_LENGTH_UPLOAD = CURLINFO_SIZE_UPLOAD

and if it does to set (via curl_setopt) CURLOPT_RESUME_FROM=CURLINFO_CONTENT_LENGTH_UPLOAD. If they are not equal, then I would restart the transfer as opposed to resuming. Does this seem like a good check?

Since I only do this transfer once in a while for backups, I thought I would run the idea by people here before experimenting with it.

Thanks.
User avatar
mecha_godzilla
Forum Contributor
Posts: 375
Joined: Wed Apr 14, 2010 4:45 pm
Location: UK

Re: PHP cURL ftp resume

Post by mecha_godzilla »

Hi,

If in doubt, check to see what commands your FTP client uses in the same situation - most FTP clients have a transcript/debug window so you can see what commands it's sending. I haven't tried using the options you mention, but there's no reason why you can't emulate the same kind of functionality with cURL.

HTH,

Mecha Godzilla
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

Re: PHP cURL ftp resume

Post by Eric! »

Thanks for the idea. It's hard to duplicate the failure on the server to server situation. I could setup something locally to verify it works. Rather than going to that trouble I thought I'd ask, but it seems there aren't many people on here that use the curl extension as much as I do. FTP and php curl FTP extensions are different so I'm not sure how to correlate them exactly.
User avatar
mecha_godzilla
Forum Contributor
Posts: 375
Joined: Wed Apr 14, 2010 4:45 pm
Location: UK

Re: PHP cURL ftp resume

Post by mecha_godzilla »

I'll see if I can write a test script over the weekend - I think the main issue is just replicating the conditions under which the transfer will fail. I don't think it matters whether the remote server stops responding or the connection is lost because cURL will probably still capture it as a time-out event, then you can reinitiate the connection again. You might want the script to give up after several attempts though, to stop it looping indefinitely.

M_G
User avatar
mecha_godzilla
Forum Contributor
Posts: 375
Joined: Wed Apr 14, 2010 4:45 pm
Location: UK

Re: PHP cURL ftp resume

Post by mecha_godzilla »

Using curl_getinfo() the output on my local machine looks something likes this:

Code: Select all

array(21) {
  ["url"]=>
  string(56) "ftp://0.0.0.0/my_file.zip"
  ["content_type"]=>
  NULL
  ["http_code"]=>
  int(350)
  ["header_size"]=>
  int(0)
  ["request_size"]=>
  int(0)
  ["filetime"]=>
  int(-1)
  ["ssl_verify_result"]=>
  int(0)
  ["redirect_count"]=>
  int(0)
  ["total_time"]=>
  float(3.064527)
  ["namelookup_time"]=>
  float(0.018974)
  ["connect_time"]=>
  float(0.358245)
  ["pretransfer_time"]=>
  float(1.986533)
  ["size_upload"]=>
  float(0)
  ["size_download"]=>
  float(0)
  ["speed_download"]=>
  float(0)
  ["speed_upload"]=>
  float(0)
  ["download_content_length"]=>
  float(314308)
  ["upload_content_length"]=>
  float(0)
  ["starttransfer_time"]=>
  float(0)
  ["redirect_time"]=>
  float(0)
  ["certinfo"]=>
  array(0) {
  }
}
Based on that output, you could just use the "download_content_length" value to compare the size of the file on the remote server with the local copy.

In my FTP client, when trying to resume a file upload the command transcript was as follows (note that the file sizes are different because these commands were run prior to resuming the upload):

Code: Select all

Cmd: TYPE I
200: TYPE is now 8-bit binary
Cmd: OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid
200:  MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
Cmd: MLST my_file.zip
250: Begin
      type=file;size=75342;modify=20130603231455;UNIX.mode=0644;UNIX.uid=2778;UNIX.gid=2779;unique=851g1609455; my_file.zip
     End.
Cmd: PORT 0,0,0,0,0,0
200: PORT command successful
Cmd: REST 75342
350: Restarting at 75342
Cmd: STOR my_file.zip
150: Connecting to port 49650
So, as long as the cURL functions work in the same way, it should be possible to resume your uploads without any problems - I've only ever had one instance where my FTP client got confused about a file's size and didn't download it correctly, so you should be safe to resume. You could call a shell script to check the backup file with MD5 or SHA1 after the transfer has been completed to be on the safe side.

M_G
Post Reply