Page 1 of 1

ssh2_scp_recv high CPU utilization

Posted: Wed Apr 04, 2007 7:07 am
by jameslabocki
I am transferring a file via SCP over a 100MB LAN connection. There are no errors on the interfaces and there is a switch between them which also shows no errors. I am saying this to ensure everyone there is no network issues. Anyway, I do something like this.

Code: Select all

if (!ssh2_scp_recv($this->connection, $this->remotefile, $this->localfile)) {
return 1;
} else {
return 0;
}
It transfers data VERY fast, but the problem is it's killing my dual CPU V210 with 2GB of RAM. Executing the script to pull just one file of 20MB across puts the CPU at 98% and everything else on the system crawls. Of course it only takes a second, but my goal is to use this script to pull several gigabytes of data across using multiple processes calling the same command. Any ideas on why it is so CPU intensive? When I copy over SCP from command line it is not killing the box as bad? I have to run 20 of these scripts concurrently throughout the day and don't want to kill the box. I also tried using fopen(ssh2.sftp), but this is extremely slow no matter what I try. Any help would be appreciated on the fastest method for getting a file via ssh2 (SFTP or SCP) as long as it's fast and doesn't take to many resources!

Posted: Wed Apr 04, 2007 7:24 am
by jameslabocki
BTW,

A truss of the process shows the following. I think it might be making too many system calls with fcntl and recv? Anyway to set the blocksize of the write without using fopen and file_get_contents? I tried those and it takes 20 minutes to transfer a 4MB file!

Code: Select all

fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "FE 4 HFB1A / J RA895 n {".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "12 6 %CB9D &D5 Y V %C7DE".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " k9FD103 pDACE049E88 {BE".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "D503B61DA1 y99 ^BE s GAD".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "0E N f = \\vE3D1C180A4C7".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "DC\b07BDE2 5 ] 2 % h1AFA".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " ~ rBB TA0 :14 |A59D 7CA".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " 58DF5 H 111\0 j ;B2 -13".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " c c85E5 / ! #80 '84 ` |".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "15 ~DBF0B71D x1FCE <B2 g".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " t1E15EFD3DDDD /88 X1EC0".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " 7A1B4D2", 16, 0)                      = 4
recv(6, " ]C6E2160E1CFF9E jC9\fA0", 12, 0)      = 12
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "80 dF2EEDE dC3D394E1841B".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "F494 wDD 6ACA5 y8287B0 y".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " " 697 &\n0F83 :E182B8 !".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "C593 '9A ?CDFFA6ACF3B4 Q".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " MC3 0E5E9 &A4 V - hE48B".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, " t0FD6D9 ~ {A2 " +D5AF1A".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "9080 A kD394C7 g r01 }8A".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "9C l vD519 S9ECBB1E8CDD7".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "8A f = # hC6 c r }BC a \".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "FD , h x9E R uF8CC14 k |".., 16, 0)    = 16
fcntl(6, F_SETFL, 0x00000000)                   = 0
recv(6, "DE9D\nD9FB1A85 eFB NFE16".., 16, 0)    = 16