Efficient file read

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
f1nutter
Forum Contributor
Posts: 125
Joined: Wed Jun 05, 2002 12:08 pm
Location: London

Efficient file read

Post by f1nutter »

Hi,

I am trying to read in some HTML files to extract anchor links. My PHP execution time is set to 30 seconds by my host, so I need to make every fraction count.

I am trying to read in about 500 pages, and time-out at about 200. I have tried to find which part of my script is the slowest and start working there. I am using

Code: Select all

<?php

$file_array = file($file_name);

?>
which reads a file into an array. This takes about 0.2 seconds on my home PC through Apache. I guess processor / hard-drive speed will effect this, but the code needs to be portable so a general increase in efficiency is needed.

Is there a quicker way to read in an HTML file from a remote location?

Cheers.
User avatar
volka
DevNet Evangelist
Posts: 8391
Joined: Tue May 07, 2002 9:48 am
Location: Berlin, ger

Post by volka »

you can use multiple sockets, set them to non-block-mode with socket_set_nonblock and read 10 maybe 20 pages at once (more or less) by writing you own parser for the http-stream.
Worth it? I don't know ;)
User avatar
hob_goblin
Forum Regular
Posts: 978
Joined: Sun Apr 28, 2002 9:53 pm
Contact:

Post by hob_goblin »

Are you using a loop?
At the end of the loop, try putting in something like set_time_limit("5");

this will give the script 5 seconds added onto the max execution time each time the loop is executed...
Post Reply