Page 1 of 1

Packing multiple files into one file. No ext libraries.

Posted: Sun Nov 23, 2008 5:39 pm
by kaisellgren
Hi,

I need to pack multiple files (from 10 to 100, varies) into one. I am not looking for a compression, just packing files.

I can not use any external libraries. I am using PHP 5.1.2 and I can only use functionality that comes with it.

Help? Ideas? Classes?

Re: Packing multiple files into one file. No ext libraries.

Posted: Sun Nov 23, 2008 7:17 pm
by Syntac
I actually tried this once. Here's a (very) basic implementation.

Code: Select all

$fp = fopen("the_pack_file.pk", "ab");
foreach($list_of_files as $file) {
    $data = base64_encode(file_get_contents($file));
    fwrite($fp, "$file\n$data\n");
}
fclose($fp);
Layout of the pack file:

Code: Select all

file1.txt
<base64-encoded data goes here>
file2.txt
<ditto>
I'm too tired to dig up the unpacking code, but it should be pretty simple. Also, you'll want to have the script automatically create any subdirectories.

Re: Packing multiple files into one file. No ext libraries.

Posted: Sun Nov 23, 2008 7:20 pm
by kaisellgren
Syntac wrote:I actually tried this once. Here's a (very) basic implementation.

Code: Select all

$fp = fopen("the_pack_file.pk", "ab");
foreach($list_of_files as $file) {
    $data = base64_encode(file_get_contents($file));
    fwrite($fp, "$file\n$data\n");
}
fclose($fp);
Layout of the pack file:

Code: Select all

file1.txt
<base64-encoded data goes here>
file2.txt
<ditto>
I'm too tired to dig up the unpacking code, but it should be pretty simple. Also, you'll want to have the script automatically create any subdirectories.
Hehe, yeah. That's simple one. I might add some changes like directory path info, etc... and base64 encoding makes the data very much longer... I wonder if there's a shorter encoding alternative.

Re: Packing multiple files into one file. No ext libraries.

Posted: Sun Nov 23, 2008 7:32 pm
by Syntac
Try escaping newlines?

[ed] Or...

Code: Select all

echo bin2hex("foo"); // 666f6f
echo pack("H*", "666f6f"); // foo
This doesn't significantly increase the filesize, but I'm sure there are better alternatives.

Re: Packing multiple files into one file. No ext libraries.

Posted: Mon Nov 24, 2008 7:08 pm
by kaisellgren
Syntac wrote:Try escaping newlines?

[ed] Or...

Code: Select all

echo bin2hex("foo"); // 666f6f
echo pack("H*", "666f6f"); // foo
This doesn't significantly increase the filesize, but I'm sure there are better alternatives.
With a filesize of 4 MB I'm receiving memory limit exceeded problems. Any help?

Re: Packing multiple files into one file. No ext libraries.

Posted: Mon Nov 24, 2008 7:14 pm
by Syntac
Okay... Maybe not that, then... :?

Re: Packing multiple files into one file. No ext libraries.

Posted: Mon Nov 24, 2008 7:39 pm
by Christopher

Re: Packing multiple files into one file. No ext libraries.

Posted: Tue Nov 25, 2008 9:59 am
by mmj
Yeah, why don't you check for zip or something?

If you only have zlib installed but not zip you can use one of the zip classes out there.

Re: Packing multiple files into one file. No ext libraries.

Posted: Tue Nov 25, 2008 12:56 pm
by kaisellgren
mmj wrote:
Yeah, why don't you check for zip or something?

If you only have zlib installed but not zip you can use one of the zip classes out there.
I need some functionality like grabbing 25% of the data of a specific file inside the pack... that's hard with those compression algos.. but let me see..

Thanks for the help guys.

Re: Packing multiple files into one file. No ext libraries.

Posted: Tue Nov 25, 2008 5:16 pm
by mmj
kaisellgren wrote:I need some functionality like grabbing 25% of the data of a specific file inside the pack...
Heh, that is a pretty specific need.

Don't think it would be possible with a compression algorithm, not sure about TAR tho.

Re: Packing multiple files into one file. No ext libraries.

Posted: Thu Dec 25, 2008 11:35 am
by Syntac
Sorry to revive this thread, but looking at how serialize() works gave me an idea: Instead of using base64 or whatever to prevent delimiter collision, just specify how long the file data is.

Re: Packing multiple files into one file. No ext libraries.

Posted: Thu Dec 25, 2008 11:42 am
by kaisellgren
Syntac wrote:Sorry to revive this thread, but looking at how serialize() works gave me an idea: Instead of using base64 or whatever to prevent delimiter collision, just specify how long the file data is.
Yep that's exactly what I'm doing :D

To be more precise for ppl that want to know the method.

I'm packing the file actual data into a .bin file (binary data), first 8 bytes specify the lenght of the upcoming data, and I hexed it, so basically i can combine up to 2^32-1 bytes of data per file.

$size = hexdec(fgets($fp,8)); // read 8 first bytes, convert to decimal to retrieve size
$content = fgets($fp,$size); // an example to read content

Ofc I store the file name paths etc, but I'm not gonna list everything here, so I just provided an exmaple what I was doing.