I need a FIFO method of FOPEN, to pop the first line ....

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
jclarkkent2003
Forum Contributor
Posts: 123
Joined: Sat Dec 04, 2004 9:14 pm

I need a FIFO method of FOPEN, to pop the first line ....

Post by jclarkkent2003 »

Hi again, heh... This is a REALLY Stupid question and hella easy I know, I just can't remeber what is the MOST efficient method of doing this....

I need to open a list.txt file and pop the first line of it into a buffer.txt file, and then when the script is finished, it finds that line in the buffer and moves to the finished.txt file.

I will be running the same script multiple instances of ( run it 20 times at any given time so the buffer.txt will have 20 lines which i will also need to know how to search for and remove that line from the file.)

The list.txt could be 1 mb in size and I prefer not to load that into the memory, but if it's possible to just pop the first line into the buffer, and not have to open the entire file, recurse though each line checking if it's the right line ?

What's an efficient way to do this ?

Thanks~
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Post by josh »

If you're not appending to the file, the only way to modify it would be to load the entire file into memory. You can however read the file into memory one line at a time or use fseek() to jump to a certain byte inside of the file.
jclarkkent2003
Forum Contributor
Posts: 123
Joined: Sat Dec 04, 2004 9:14 pm

Post by jclarkkent2003 »

right now my method is this:

Code: Select all

<?

$filename = file("list.txt");
$first_line = array_shift($filename);

$handle = fopen{"list.txt","w+");
fwrite($handle,$filename);
fclose($handle);

$handle = fopen{"buffer.txt","a+");
fwrite($handle,$first_line);
fclose($handle);


and then when the file is done running....

$filename = file("buffer.txt");

for($k=0;$k<sizeof($filename);$k++)
{

if($filename[$k] == $firstline) { unset($filename[$k]); // not sure if this one will work for a variable inside an array , never tried. }

}

$buffer_removed_line = implode("\n",$filename);

$handle = fopen{"buffer.txt","w+");
fwrite($handle,$buffer_removed_line);
fclose($handle);


$handle = fopen{"finished.txt","a+");
fwrite($handle,$firstline);
fclose($handle);

?>
K, i didn't test the above but it's my raw coding for what I need to do. Is there a more efficient way, as like i said, say the list.txt can be upto 100 megs in size, and the buffer.txt be 50,000 lines, and the finished.txt be 300 mb

What is a more efficient way of doing this?
jclarkkent2003
Forum Contributor
Posts: 123
Joined: Sat Dec 04, 2004 9:14 pm

Post by jclarkkent2003 »

jshpro2 wrote:If you're not appending to the file, the only way to modify it would be to load the entire file into memory. You can however read the file into memory one line at a time or use fseek() to jump to a certain byte inside of the file.
See, Fseek would be perfect, but how can I then remove that line from list.txt ? I ONLY need to appened the FIRST line somehow the easiest fastest and most efficient way possible....
jclarkkent2003
Forum Contributor
Posts: 123
Joined: Sat Dec 04, 2004 9:14 pm

Post by jclarkkent2003 »

any ideas my brothers/sista's?

Efficiency is going to be a BIG deal with this script once it's more complete. Just two things left, this one to make it more efficient and the other one is my get page through proxy, then it's 90% complete.

Thanks in advance~
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

use a database.

There is no simple and efficient way of FIFO'ing a large file.
jclarkkent2003
Forum Contributor
Posts: 123
Joined: Sat Dec 04, 2004 9:14 pm

Post by jclarkkent2003 »

yuck!!! lol, i want to stay clear of a database.

So your saying there is NO possible function or method I can fseek the first line, and somehow remove that first line as well without opening the full file ?

:(

So my method above is the ONLY possible way to do it? ...

Involving a mysql database would eat up the load like crazy and put strain on the server for the tasks i'm planning for this script. flat file db is alot easier on the server as there would be way too many COMPLEX mysql queries going... Mysql is already handling enough as is, I hate it for sake of shooting my server's load sky high ( that's what you get when you host thousands of sites on one server.)
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Post by josh »

mysql is generally going to be more cost efficient then reading the entire file into memory and then writing it back again. I recommend running benchmarks on both methods
jclarkkent2003
Forum Contributor
Posts: 123
Joined: Sat Dec 04, 2004 9:14 pm

Post by jclarkkent2003 »

oh i have ran benchmarks, heh, flat file is barely touches my load where as mysql eats it up like crazy until it's done, like i said, it's like that because there's aLREADY a few thosuand sites on the cpanel server using the mysql databases, mysql can only handle so much and it's there.
Post Reply