Write file block by block

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
messier79
Forum Newbie
Posts: 4
Joined: Sun Oct 31, 2010 1:49 pm

Write file block by block

Post by messier79 »

Hi

In my script, I receive a file splitted in blocks and I would want to reconstitute the file writing block by block.
I did a fopen on a file and used fwrite to write the content. It works, but, when the total size exceeds the value of memory_limit, it crashes.
I would want to find a way to write the file directly on the disk to avoid that.
Of course, I could simply increase the value of memory_limit, but I could work with very large files and that's not very good.

Any idea ?

Thanks
gooney0
Forum Commoner
Posts: 56
Joined: Fri Jan 21, 2011 1:40 pm
Location: Reston, VA

Re: Write file block by block

Post by gooney0 »

A few ideas which may or may not actually work. 8O

1) fopen and fclose the file every xx lines. Maybe unset the data as it's written too.

2) Create a program which only handles X number of lines and saves it progress as a line number. When run the program gets the line # from $_SESSION and processes 10,000 lines then saves the new line number to $_SESSION. You could then use a parent program to use ajax to run the program many times while updating the user on it's progress.

The logic for #2 looks kinda like this:

Parent page:
Javascript loop every 20 seconds to child.php

child.php
Process 1,000 lines save ending line to $_SESSION["progress"]
Save status to $_SESSION["status"]
When done execute javascript function to stop the parent and display some output.
User avatar
requinix
Spammer :|
Posts: 6617
Joined: Wed Oct 15, 2008 2:35 am
Location: WA, USA

Re: Write file block by block

Post by requinix »

The memory limit comes into play when you receive the file data, not when you try to write it out.
Receive only a portion of it, write it out, rinse and repeat.
Post Reply