Page 1 of 1

Help with cURL FTP SSL Connections

Posted: Sat Feb 23, 2013 5:59 pm
by mecha_godzilla
Hi All,

I hope someone can help me with this problem please:

I'm trying to use cURL to transfer a file from my machine to a remote server using SSL but I just keep getting time-outs. I can connect to the server fine using a separate application (Cyberduck) but I've tried all different combinations of cURL options without success and the only error message that gets reported is 35 (CURLE_SSL_CONNECT_ERROR).

According to Cyberduck, the SSL connection it uses is "Explicit Auth TLS" and when I log in I get this information:

Code: Select all

220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
220-You are user number 6 of 1000 allowed.
AUTH TLS
234 AUTH TLS OK.
USER myusername
331 User myusername OK. Password required
PASS ********
230 OK. Current restricted directory is /
PBSZ 0
200 PBSZ=0
PROT P
200 Data protection level set to "private"
I get a warning about the certificate not being valid but this is because I have to connect using the server's IP address rather than a domain name, which can be handled in the cURL options.

In case this information is relevant, I'm using PHP v5.3.5 and have cURL v7.21.3 and here is my code - I'm not trying to transfer any files here, just get a directory listing:

Code: Select all

$ftp_server = 'ftps://255.255.255.255:21/';

$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $ftp_server);
curl_setopt($curl, CURLOPT_FTPLISTONLY, 1);
curl_setopt($curl, CURLOPT_USERPWD, "myusername:mypassword");
curl_setopt($curl, CURLOPT_VERBOSE, TRUE);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, FALSE); 
curl_setopt($curl, CURLOPT_FTP_SSL, CURLFTPSSL_ALL);
curl_setopt($curl, CURLOPT_FTPSSLAUTH, CURLFTPAUTH_TLS);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 10);
$result = curl_exec($curl);
$error_no = curl_errno($curl);

if ($error_no == 0) {
	echo 'Connection made';
} else {
	echo 'Connection failed: ' . $error_no;
}

curl_close($curl);
Thanks in advance,

Mecha Godzilla

Re: Help with cURL FTP SSL Connections

Posted: Sat Feb 23, 2013 6:47 pm
by requinix
There are a few causes for error 35. What is the exact error message?

And side note, I assume you're not actually trying to connect to 255.255.255.255 right?

Re: Help with cURL FTP SSL Connections

Posted: Sat Feb 23, 2013 7:27 pm
by mecha_godzilla
Thanks for replying so quickly :mrgreen:

The 255.255.255.255 address was just a smoke screen, btw.

I added the stderr option and got this output:

Code: Select all

* About to connect() to 255.255.255.255 port 21 (#0)
*   Trying 255.255.255.255... * connected
* Connected to 255.255.255.255 (255.255.255.255) port 21 (#0)
* error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
* Closing connection #0
So I then tried running curl directly from the command line - I get the same kind of error message but if I do this

$ curl -2 -v --user 'myusername:mypassword' ftps://255.255.255.255:21

then I get as far as this

Code: Select all

* About to connect() to 255.255.255.255 port 21 (#0)
*   Trying 255.255.255.255... connected
* Connected to 255.255.255.255 (255.255.255.255) port 21 (#0)
* successfully set certificate verify locations:
*   CAfile: /usr/share/curl/curl-ca-bundle.crt
  CApath: none
* SSLv2, Client hello (1):
and it then just waits and does nothing and eventually I get the time-out message. If I use -1 or -3 instead I get

Code: Select all

* About to connect() to 255.255.255.255 port 21 (#0)
*   Trying 255.255.255.255... connected
* Connected to 255.255.255.255 (255.255.255.255) port 21 (#0)
* successfully set certificate verify locations:
*   CAfile: /usr/share/curl/curl-ca-bundle.crt
  CApath: none
 SSLv3, TLS handshake, Client hello (1):
* SSLv?, Unknown (2):
* error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
* Closing connection #0
curl: (35) error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
I must admit I'm confused - it's saying "SSLv3, TLS handshake" and the curl manual says -1 is for TLS.

Thanks again,

M_G

Re: Help with cURL FTP SSL Connections

Posted: Sat Feb 23, 2013 7:56 pm
by mecha_godzilla
Hi again,

I've just tried using ftp_ssl_connect() and it seems to be working - I can get a directory listing now and have just uploaded a file. Do you know if there are any advantages or disadvantages using this function as opposed to cURL? I wonder what PHP is doing that cURL isn't - or maybe it's just operator error at my end...

At least there's some progress anyway - I have to get this script finished and tested for first thing Monday morning :dubious:

M_G

Re: Help with cURL FTP SSL Connections

Posted: Sat Feb 23, 2013 8:17 pm
by requinix
So there's implicit FTPS where the whole connection is secure, and explicit where the FTP client asks for SSL after the connection is established and as part of the FTP transaction. I think cURL is trying to do implicit FTPS, but what I've seen suggests (not very strongly) that ftps:// should trigger explicit FTPS. On the command line you could use --ftp-ssl (deprecated in favor of --ssl) to force explicit mode but I haven't seen a corresponding option for curl_setopt().
You could packet-sniff to find out exactly what cURL is doing.

cURL in general is just more flexible at how connections and operations are done. Personally if there's a built-in function that does what I need I would rather use that.

Re: Help with cURL FTP SSL Connections

Posted: Sat Feb 23, 2013 8:57 pm
by mecha_godzilla
I think you're definitely right about the issue with cURL being the explicit/implicit thing - I just tried using ftp:// rather than ftps:// like this

Code: Select all

$ curl --ftp-ssl --insecure -3 -v --user 'myusername:mypassword' ftp://255.255.255.255:21
and the TLS "handshake" actually completes but I then got stuck at

Code: Select all

< 230 OK. Current restricted directory is /
> PBSZ 0
< 200 PBSZ=0
> PROT P
< 200 Data protection level set to "private"
> PWD
< 257 "/" is your current location
* Entry path is '/'
> PASV
* Connect data stream passively
< 227 Entering Passive Mode (255,255,255,255,255,255)
* Skips 255.255.255.255 for data connection, uses 255.255.255.255 instead
*   Trying 255.255.255.255... Operation timed out
* couldn't connect to host
* Closing connection #0
* SSLv3, TLS alert, Client hello (1)
so I then did this

Code: Select all

$ curl --ftp-ssl --insecure -3 -v --user 'myusername:mypassword' --ftp-port - ftp://255.255.255.255:21
and - as if by magic - I get logged-in, everything looks good and I get a directory listing :mrgreen: Now I just have to get it working in the script, or probably cheat and use exec() if I'm feeling brave...

Thanks again for your help - it's just coming up to 3am here so I'm done for the night!

M_G

Re: Help with cURL FTP SSL Connections

Posted: Sun Feb 24, 2013 3:10 pm
by mecha_godzilla
Hi again,

For the benefit of anyone else that might be having this kind of problem, here is an in-script version of the command line statement above:

Code: Select all

$ftp_server = 'ftp://255.255.255.255:21/';

$curl = curl_init();

$stderr = fopen("curl.txt", "w"); 

curl_setopt($curl, CURLOPT_URL, $ftp_server);
curl_setopt($curl, CURLOPT_FTPLISTONLY, 1);
curl_setopt($curl, CURLOPT_USERPWD, "myusername:mypassword");
curl_setopt($curl, CURLOPT_VERBOSE, TRUE);
curl_setopt($curl, CURLOPT_STDERR, $stderr);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($curl, CURLOPT_FTP_SSL, CURLFTPSSL_ALL);
curl_setopt($curl, CURLOPT_FTPSSLAUTH, CURLFTPAUTH_TLS);
curl_setopt($curl, CURLOPT_SSLVERSION, 3);
curl_setopt($curl, CURLOPT_FTPPORT,'-');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 30);

$result = curl_exec($curl);

$error_no = curl_errno($curl);

if ($error_no == 0) {
	echo 'Connection made';
} else {
	echo 'Connection failed: ' . $error_no;
}

var_dump($result);
curl_close($curl);
fclose($stderr);
The CURLOPT_FTPPORT option is probably only required if you're having issues with PASV mode FTP connections (which I always seem to, and this is different to the CURLOPT_PORT option that I was mistakenly using) and note that using the CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST options are not recommended for use under normal circumstances.

HTH,

M_G

Re: Help with cURL FTP SSL Connections

Posted: Sun Feb 24, 2013 4:37 pm
by Christopher
So if I understand the change you made, you set CURLOPT_FTPPORT to '-'. I read the section "FTP Peculiarities We Need" here (http://curl.haxx.se/libcurl/c/libcurl-tutorial.html) and it indicates that libcurl will use your system's "default IP address".

Re: Help with cURL FTP SSL Connections

Posted: Sun Feb 24, 2013 5:57 pm
by mecha_godzilla
Yes, that's correct - I'm not going to pretend I really know how that all works, but passive mode FTP has never worked with my internet connection - I used to think it was a problem with my firewall settings until I switched it off and it still didn't work in any of my FTP clients, so it must be to do with my ISP's arrangements. I happened to stumble across the '-' mode by accident as I was reading the cURL manual, and I just tried setting the CURLOPT_FTP_USE_EPRT option to zero as mentioned in the link you posted and it does work - the connections both worked fine, and you can see in the edited logs below that it was sending the appropriate commands

Code: Select all

> EPRT |1|000.000.000.000|49505|
< 200 PORT command successful
* Connect data stream actively
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR mydocument.pdf
< 150 Connecting to port 49505
and

Code: Select all

> PORT 000,000,000,000,193,221
< 200 PORT command successful
* Connect data stream actively
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR mydocument.pdf
< 150 Connecting to port 49629
with 000.000.000.000 being a placeholder for my real IP address and the two numbers at the end of the PORT command somehow translating to the correct port number.

So, to briefly summarise what I've learned:

1. The SSL handshake was failing outright because I had specified "ftps://" rather than "ftp://".
2. I was also using the wrong SSL version (2 rather than 3, but I had no real way of knowing that except by trial-and-error).
3. The connection was then timing-out after the SSL handshake and connection had been successfully negotiated because it was using an invalid port for my machine in active mode, then when it switched to passive mode it didn't work because my internet connection didn't like it.

I have this all working reliably in my script now, along with an implementation using ftp_ssl_connect(). The ftp_ssl_connect() way seems much easier (I just have to specify a different port number for the server and no other custom settings) and there's no obvious difference in performance, so I guess the moral there is just to use what works :mrgreen:

Thanks again for everyone's help and interest,

M_G