Page 1 of 1

fread() fetching only 8K of data

Posted: Mon Jul 30, 2007 12:44 am
by rishi
I am using fread() function to fetch web page contents, but it is not taking more then 8K data at a time.
I am using PHP 5. here I am opening Multiple sockets to get page contents.
Following is my code for getting page contents.

Code: Select all

$hosts = array("example1.com","example2.com");

foreach ($hosts as $id => $host) {
                $s = stream_socket_client("$host:80", $errno, $errstr, $timeout,
                STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT);
       
                if ($s) {
                        $sockets[$id] = $s;
                        $status[$id] = "in progress";
               
                } else {
                        $status[$id] = "failed, $errno $errstr";
                }
        }

  while (count($sockets)) {
                $read = $write = $sockets;
                $n = stream_select($read, $write, $e = null, $timeout);
               
                if ($n > 0) {
                        foreach ($read as $r) {
                                $id = array_search($r, $sockets);
                                $data = fread($r,8194);     
                                               
                                if (strlen($data) == 0) {
                                if ($status[$id] == "in progress") {
                                        $status[$id] = "failed to connect";
                                }
                                fclose($r);
                                unset($sockets[$id]);
                                } else {
                                        $status[$id] .= $data;
                                }
                  }
here i have opened "n" no. of sockets for "n" no. of URL's , they are in array $sockets.
I want to fetch page content of all the pages simultaneously.
I tried giving more then 8K here but not seems to look working

Code: Select all

$data = fread($r,8194);
any suggestions.
thanks in advance.

Posted: Mon Jul 30, 2007 12:49 am
by John Cartwright
Hmm, the manually is glorious :wink:
string fread ( resource $handle, nt $length )

fread() reads up to length bytes from the file pointer referenced by handle. Reading stops when up to length bytes have been read,
You have only specified 8K as the length.. filesize() may be of interest

Posted: Mon Jul 30, 2007 3:47 am
by rishi
yes, i have tried giving more then 8K also :( .

Posted: Mon Jul 30, 2007 4:03 am
by Chris Corbyn
The manual wrote:fread() reads up to length bytes from the file pointer referenced by handle. Reading stops when up to length bytes have been read, EOF (end of file) is reached, (for network streams) when a packet becomes available, or (after opening userspace stream) when 8192 bytes have been read whichever comes first.
You have to iterate (with a while loop?) with fread() if you want the lot. The IO stream cannot buffer an entire file in one go.