Creating Large ZIP Files

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
arkady
Forum Newbie
Posts: 23
Joined: Sun Sep 17, 2006 9:34 pm

Creating Large ZIP Files

Post by arkady »

Gday Guys, i'm currently using a brilliant class by Eric Mueller (modified) http://www.zend.com/codex.php?id=696&single=1 to create zip files 'on the fly'. Basically, i retrieve a file from a read only location, dump it into a string add it to the class, create the zip then send it to the client with the appropriate headers.

Simple stuff and it works great, when using files of a smaller nature. The problem that I have is that I'm using highly compressible, yet very large files (Autocad Drawings) and I run into memory & script time constraints. When I say large, some files that i'm using could be upwards of 300mb which will obviously use a large memory footprint, and will take some time (>60seconds).

At present, i've dropped a script timeout when creating these zip's, and have allowed the server to use up to around 600mb of memory to 'temporarily get around this problem'. Obviously this is not going to be feasible when the server goes live.

What I want, is a 'chunkable' zip file creator where I could 'feed' arbitrary amounts of the file (say <50mb) at a time, to create the zip file using a smaller memory footprint. Assuming that for the time being, time is no major constraint (if the client has to wait 3-4minutes for the file to compress, it's far better than the additional 2 hours to download the extra 100mb), has anyone enountered a class, or ZIP method that would allow me to do this? Failing that, does anyone know of a zip algorithm/format that would allow me to begin encoding a file without requiring the entire file loaded into memory?

Cheers
User avatar
akimm
Forum Contributor
Posts: 460
Joined: Thu Apr 27, 2006 10:50 am
Location: Ypsilanti Michigan, formally Clipsburgh

Post by akimm »

Hey.
This may be of interest :D

viewtopic.php?t=29066

Always check code snipplets, if it doesn't suite your purposes sorry ^_^
arkady
Forum Newbie
Posts: 23
Joined: Sun Sep 17, 2006 9:34 pm

Post by arkady »

Appreciate the thought akimm, alas it's not quite what i'm after.

I'm attempting to create ZIP files to deliver to my users on the fly. Smaller files work great, as do larger ones (but with significant memory usage). I need to find a way to minimise the memory footprint used by the zip creation process assuming time is no object.
Post Reply