Page 1 of 1

Write file block by block

Posted: Sun Jan 30, 2011 3:40 pm
by messier79
Hi

In my script, I receive a file splitted in blocks and I would want to reconstitute the file writing block by block.
I did a fopen on a file and used fwrite to write the content. It works, but, when the total size exceeds the value of memory_limit, it crashes.
I would want to find a way to write the file directly on the disk to avoid that.
Of course, I could simply increase the value of memory_limit, but I could work with very large files and that's not very good.

Any idea ?

Thanks

Re: Write file block by block

Posted: Sun Jan 30, 2011 7:14 pm
by gooney0
A few ideas which may or may not actually work. 8O

1) fopen and fclose the file every xx lines. Maybe unset the data as it's written too.

2) Create a program which only handles X number of lines and saves it progress as a line number. When run the program gets the line # from $_SESSION and processes 10,000 lines then saves the new line number to $_SESSION. You could then use a parent program to use ajax to run the program many times while updating the user on it's progress.

The logic for #2 looks kinda like this:

Parent page:
Javascript loop every 20 seconds to child.php

child.php
Process 1,000 lines save ending line to $_SESSION["progress"]
Save status to $_SESSION["status"]
When done execute javascript function to stop the parent and display some output.

Re: Write file block by block

Posted: Sun Jan 30, 2011 11:17 pm
by requinix
The memory limit comes into play when you receive the file data, not when you try to write it out.
Receive only a portion of it, write it out, rinse and repeat.