Packing multiple files into one file. No ext libraries.

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Packing multiple files into one file. No ext libraries.

Post by kaisellgren »

Hi,

I need to pack multiple files (from 10 to 100, varies) into one. I am not looking for a compression, just packing files.

I can not use any external libraries. I am using PHP 5.1.2 and I can only use functionality that comes with it.

Help? Ideas? Classes?
User avatar
Syntac
Forum Contributor
Posts: 327
Joined: Sun Sep 14, 2008 7:59 pm

Re: Packing multiple files into one file. No ext libraries.

Post by Syntac »

I actually tried this once. Here's a (very) basic implementation.

Code: Select all

$fp = fopen("the_pack_file.pk", "ab");
foreach($list_of_files as $file) {
    $data = base64_encode(file_get_contents($file));
    fwrite($fp, "$file\n$data\n");
}
fclose($fp);
Layout of the pack file:

Code: Select all

file1.txt
<base64-encoded data goes here>
file2.txt
<ditto>
I'm too tired to dig up the unpacking code, but it should be pretty simple. Also, you'll want to have the script automatically create any subdirectories.
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Packing multiple files into one file. No ext libraries.

Post by kaisellgren »

Syntac wrote:I actually tried this once. Here's a (very) basic implementation.

Code: Select all

$fp = fopen("the_pack_file.pk", "ab");
foreach($list_of_files as $file) {
    $data = base64_encode(file_get_contents($file));
    fwrite($fp, "$file\n$data\n");
}
fclose($fp);
Layout of the pack file:

Code: Select all

file1.txt
<base64-encoded data goes here>
file2.txt
<ditto>
I'm too tired to dig up the unpacking code, but it should be pretty simple. Also, you'll want to have the script automatically create any subdirectories.
Hehe, yeah. That's simple one. I might add some changes like directory path info, etc... and base64 encoding makes the data very much longer... I wonder if there's a shorter encoding alternative.
User avatar
Syntac
Forum Contributor
Posts: 327
Joined: Sun Sep 14, 2008 7:59 pm

Re: Packing multiple files into one file. No ext libraries.

Post by Syntac »

Try escaping newlines?

[ed] Or...

Code: Select all

echo bin2hex("foo"); // 666f6f
echo pack("H*", "666f6f"); // foo
This doesn't significantly increase the filesize, but I'm sure there are better alternatives.
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Packing multiple files into one file. No ext libraries.

Post by kaisellgren »

Syntac wrote:Try escaping newlines?

[ed] Or...

Code: Select all

echo bin2hex("foo"); // 666f6f
echo pack("H*", "666f6f"); // foo
This doesn't significantly increase the filesize, but I'm sure there are better alternatives.
With a filesize of 4 MB I'm receiving memory limit exceeded problems. Any help?
User avatar
Syntac
Forum Contributor
Posts: 327
Joined: Sun Sep 14, 2008 7:59 pm

Re: Packing multiple files into one file. No ext libraries.

Post by Syntac »

Okay... Maybe not that, then... :?
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Packing multiple files into one file. No ext libraries.

Post by Christopher »

(#10850)
mmj
Forum Contributor
Posts: 118
Joined: Fri Oct 31, 2008 4:00 pm

Re: Packing multiple files into one file. No ext libraries.

Post by mmj »

Yeah, why don't you check for zip or something?

If you only have zlib installed but not zip you can use one of the zip classes out there.
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Packing multiple files into one file. No ext libraries.

Post by kaisellgren »

mmj wrote:
Yeah, why don't you check for zip or something?

If you only have zlib installed but not zip you can use one of the zip classes out there.
I need some functionality like grabbing 25% of the data of a specific file inside the pack... that's hard with those compression algos.. but let me see..

Thanks for the help guys.
mmj
Forum Contributor
Posts: 118
Joined: Fri Oct 31, 2008 4:00 pm

Re: Packing multiple files into one file. No ext libraries.

Post by mmj »

kaisellgren wrote:I need some functionality like grabbing 25% of the data of a specific file inside the pack...
Heh, that is a pretty specific need.

Don't think it would be possible with a compression algorithm, not sure about TAR tho.
User avatar
Syntac
Forum Contributor
Posts: 327
Joined: Sun Sep 14, 2008 7:59 pm

Re: Packing multiple files into one file. No ext libraries.

Post by Syntac »

Sorry to revive this thread, but looking at how serialize() works gave me an idea: Instead of using base64 or whatever to prevent delimiter collision, just specify how long the file data is.
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Packing multiple files into one file. No ext libraries.

Post by kaisellgren »

Syntac wrote:Sorry to revive this thread, but looking at how serialize() works gave me an idea: Instead of using base64 or whatever to prevent delimiter collision, just specify how long the file data is.
Yep that's exactly what I'm doing :D

To be more precise for ppl that want to know the method.

I'm packing the file actual data into a .bin file (binary data), first 8 bytes specify the lenght of the upcoming data, and I hexed it, so basically i can combine up to 2^32-1 bytes of data per file.

$size = hexdec(fgets($fp,8)); // read 8 first bytes, convert to decimal to retrieve size
$content = fgets($fp,$size); // an example to read content

Ofc I store the file name paths etc, but I'm not gonna list everything here, so I just provided an exmaple what I was doing.
Post Reply