Hi
In my script, I receive a file splitted in blocks and I would want to reconstitute the file writing block by block.
I did a fopen on a file and used fwrite to write the content. It works, but, when the total size exceeds the value of memory_limit, it crashes.
I would want to find a way to write the file directly on the disk to avoid that.
Of course, I could simply increase the value of memory_limit, but I could work with very large files and that's not very good.
Any idea ?
Thanks
Write file block by block
Moderator: General Moderators
Re: Write file block by block
A few ideas which may or may not actually work.
1) fopen and fclose the file every xx lines. Maybe unset the data as it's written too.
2) Create a program which only handles X number of lines and saves it progress as a line number. When run the program gets the line # from $_SESSION and processes 10,000 lines then saves the new line number to $_SESSION. You could then use a parent program to use ajax to run the program many times while updating the user on it's progress.
The logic for #2 looks kinda like this:
Parent page:
Javascript loop every 20 seconds to child.php
child.php
Process 1,000 lines save ending line to $_SESSION["progress"]
Save status to $_SESSION["status"]
When done execute javascript function to stop the parent and display some output.
1) fopen and fclose the file every xx lines. Maybe unset the data as it's written too.
2) Create a program which only handles X number of lines and saves it progress as a line number. When run the program gets the line # from $_SESSION and processes 10,000 lines then saves the new line number to $_SESSION. You could then use a parent program to use ajax to run the program many times while updating the user on it's progress.
The logic for #2 looks kinda like this:
Parent page:
Javascript loop every 20 seconds to child.php
child.php
Process 1,000 lines save ending line to $_SESSION["progress"]
Save status to $_SESSION["status"]
When done execute javascript function to stop the parent and display some output.
Re: Write file block by block
The memory limit comes into play when you receive the file data, not when you try to write it out.
Receive only a portion of it, write it out, rinse and repeat.
Receive only a portion of it, write it out, rinse and repeat.