Writing to a file till a size is reached

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Writing to a file till a size is reached

Post by kkonline »

Hi,
I am writing to a file using fopen and other commands to keep writing some contents to that file UNTIL A PARTICULAR SIZE ex 40KB is reached.

The max size of the file should be around 40kb; how can I perform this task

I wrote

Code: Select all

 
$snoopy = new Snoopy;
 
$myFile = "testfile.txt";
echo (filesize($myFile));   
while (filesize($myFile)<40000){
echo "file opened";
$fh = fopen($myFile, 'a') or die("can't open file");
 
    $url="kkonline.org";
$snoopy->fetchtext($url);
$resulttext= $snoopy->results;
 
$stringData = $resulttext;
fwrite($fh, $stringData);
$stringData = "\n";
fwrite($fh, $stringData);
fclose($fh);
echo $resulttext;
}
 
But not getting the desired results.
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Writing to a file till a size is reached

Post by VladSun »

Weird ...
It seems that fclose() doesn't flush the file indeed... It's flushed after the script has finished its execution. Nasty :)
There are 10 types of people in this world, those who understand binary and those who don't
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Re: Writing to a file till a size is reached

Post by kkonline »

Got it working

Code: Select all

$fh = fopen($myFile, 'a') or die("can't open file");
 
// needed otherwise filesize() may return wrong size
clearstatcache();
 
// calc how many bytes we can write
$remaining_bytes = 40000 - filesize($myFile);
 
// truncate our data if its too big
if (strlen($resulttext) > $remaining_bytes) {
    $resulttext = substr($resulttext, 0, $remaining_bytes);
}
 
fwrite($fh, $resulttext); 
 
I am thinking of creating files dynamically; writing 40KB of data in each and the remaining data on the resulttext to "myfile".$i.".txt"

such that myfile1.txt with 40KB data then the data will continue on myfile2.txt and on reaching 40KB limit will be written on myfile3.txt.

Is this possible and how can I implement this; any idea?
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Writing to a file till a size is reached

Post by VladSun »

Heh, I can't belive I missed the "Notice" section in filesize() man page :)

A quick question - what's the main idea of having these multiple files? What's the reason?
There are 10 types of people in this world, those who understand binary and those who don't
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Re: Writing to a file till a size is reached

Post by kkonline »

Basically I am developing a simple logic for displaying the feeds; however the file grows at a very fast rate and because of which it doesn't get indexed (due to big file; loading time increases) so I am creating a number of files each with say 40kb of data only.
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Writing to a file till a size is reached

Post by VladSun »

untested!

Code: Select all

// return the PART_IX of the last file used
function write_feed($file_name, $content, $part_ix = 0, $max_length = 4000)
{
    clearstatcache();
    
    $finished = false;
    
    while (!$finished)
    {
        $full_file_name = $file_name.($part_ix == 0 ? '.txt' : '.'.$part_ix.'.txt');
        $handle = @fopen($full_file_name, 'a+') or die("can't open file ".$full_file_name);
 
        $remaining_bytes = $max_length - filesize($full_file_name);
        if (strlen($content) > $remaining_bytes)
        { 
            $partial_content = substr($content, 0, $remaining_bytes);
            $content = substr($content, $remaining_bytes + 1);
            $part_ix ++;
        }
        else
        {
            $partial_content = $content;
            $finished = true;
        }
         
        fwrite($handle, $partial_content);
        fclose($handle);
    }
    return $part_ix;
}
Last edited by VladSun on Fri Oct 10, 2008 11:09 am, edited 3 times in total.
There are 10 types of people in this world, those who understand binary and those who don't
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Re: Writing to a file till a size is reached

Post by kkonline »

Hi,

Code: Select all

$snoopy = new Snoopy;
 
$snoopy->fetchtext($url);
$resulttext= $snoopy->results;
 
$myFile = "testfile.txt";
 
echo "file opened";
$fh = fopen($myFile, 'a') or die("can't open file");
 
// needed otherwise filesize() may return wrong size
 
clearstatcache();
 
// calc how many bytes we can write
 
$remaining_bytes = 5000 - filesize($myFile);
 
// truncate our data if its too big
 
if (strlen($resulttext) > $remaining_bytes) {
 
    $resulttext = substr($resulttext, 0, $remaining_bytes);
 
}
 
//fwrite($fh, $resulttext); 
 
//////////////////////////
 
//search and replace
$patterns[0] = '/"/';
$patterns[1] = '/Comments/';
$patterns[2] = '/Have something you would like to share?/';
$replacements[2] = '<p>';
$replacements[1] = '';
$replacements[0] = '';
$replaced=preg_replace($patterns, $replacements, $resulttext);
//search and replace
 
$stringData = $replaced;
fwrite($fh, $stringData);
$stringData = "\n";
fwrite($fh, $stringData);
fclose($fh);
//echo $replaced;
 
////////////////
 // return the PART_IX of the last file used
 function write_feed($file_name, $content, $part_ix = 0, $max_length = 4000)
 {
     clearstatcache();
    
     $finished = false;
    
     while (!$finished)
     {
         $file_name .= ($part_ix == 0 ? '.txt' : '.'.$part_ix.'.txt');
         $handle = @fopen($file_name, 'a+') or die("can't open file ".$file_name);
 
         $remaining_bytes = $max_length - filesize($file_name);
         if (strlen($content) > $remaining_bytes)
         {
             $partial_content = substr($contents, 0, $remaining_bytes);
             $content = substr($contents, $remaining_bytes + 1);
             $part_ix ++;
         }
         else
            $finished = true;
        
          
         fwrite($handle, $content);
         fclose($handle);
     }
     return $part_ix;
}
 
$last=write_feed($myFile,$replaced,0,4000);
echo $last;
The code above ;
1> The files are created in this fashion testfile.txt then testfile.txt.txt then testfile.txt.txt.1.txt
2> The output is

Code: Select all

Quotes
 
file opened
Notice: Undefined variable: contents in c:\program files\easyphp\www\snoopy\index_v2.php on line 87
 
Notice: Undefined variable: contents in c:\program files\easyphp\www\snoopy\index_v2.php on line 88
1
3> Only the forst file testfile.txt has contents rest all are blank
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Writing to a file till a size is reached

Post by VladSun »

:) You've tested it, I've edited it :)
There are 10 types of people in this world, those who understand binary and those who don't
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Re: Writing to a file till a size is reached

Post by kkonline »

Code I tested is

Code: Select all

$snoopy = new Snoopy;
 
$snoopy->fetchtext($url);
$resulttext= $snoopy->results;
 
$myFile = "testfile";
 
echo "file opened";
$fh = fopen($myFile, 'a') or die("can't open file");
 
// needed otherwise filesize() may return wrong size
 
clearstatcache();
 
// calc how many bytes we can write
 
$remaining_bytes = 5000 - filesize($myFile);
 
// truncate our data if its too big
 
if (strlen($resulttext) > $remaining_bytes) {
 
    $resulttext = substr($resulttext, 0, $remaining_bytes);
 
}
 
//fwrite($fh, $resulttext); 
 
//////////////////////////
 
//search and replace
$patterns[0] = '/"/';
$patterns[1] = '/Comments/';
$patterns[2] = '/Have something you would like to share?/';
$replacements[2] = '<p>';
$replacements[1] = '';
$replacements[0] = '';
$replaced=preg_replace($patterns, $replacements, $resulttext);
//search and replace
 
$stringData = $replaced;
fwrite($fh, $stringData);
$stringData = "\n";
fwrite($fh, $stringData);
fclose($fh);
//echo $replaced;
 
////////////////
   // return the PART_IX of the last file used
    function write_feed($file_name, $content, $part_ix = 0, $max_length = 2000)
    {
        clearstatcache();
      
        $finished = false;
       
        while (!$finished)
        {
           $full_file_name = $file_name.($part_ix == 0 ? '.txt' : '.'.$part_ix.'.txt');
           $handle = @fopen($full_file_name, 'a+') or die("can't open file ".$full_file_name);
    
           $remaining_bytes = $max_length - filesize($full_file_name);
           if (strlen($content) > $remaining_bytes)
           {
              $partial_content = substr($content, 0, $remaining_bytes);
               $content = substr($content, $remaining_bytes + 1);
              $part_ix ++;
         }
         else
              $finished = true;
         
           
          fwrite($handle, $content);
           fclose($handle);
       }
       return $part_ix;
   }
 
$last=write_feed($myFile,$replaced,0,4000);
echo $last;
I tested this
1> The first file has contents of 4 kb ...ok
2> the second file has the contents which are repeated from the contents of 1st file
3>the third file also has the contents which are repeated from the contents of 1st file.

means 2nd and 3rd file have duplicate contents which are indeed repeated contents from 1st fileHello again,
Last edited by kkonline on Fri Oct 10, 2008 11:29 am, edited 2 times in total.
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Writing to a file till a size is reached

Post by VladSun »

Test

Code: Select all

$s = '';
for ($i=0; $i < 20; $i++)
    $s .= str_repeat($i, 200);
    
echo strlen($s);
 
write_feed('test', $s, 0, 200);
And try to fix errors, if there are some ;)
There are 10 types of people in this world, those who understand binary and those who don't
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Re: Writing to a file till a size is reached

Post by kkonline »

VladSun wrote:Test

Code: Select all

$s = '';
for ($i=0; $i < 20; $i++)
    $s .= str_repeat($i, 200);
    
echo strlen($s);
 
write_feed('test', $s, 0, 200);
And try to fix errors, if there are some ;)
The first file has 1 to 19
the second file has 2 to 19
the third has 3 to 19
and so on...


any suggestions
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Writing to a file till a size is reached

Post by VladSun »

Yes - one.
Use my corrected code, not the second-edit one ;)
There are 10 types of people in this world, those who understand binary and those who don't
kkonline
Forum Contributor
Posts: 251
Joined: Thu Aug 16, 2007 12:54 am

Re: Writing to a file till a size is reached

Post by kkonline »

Just one small change

Code: Select all

$partial_content = substr($content, 0, $remaining_bytes);
$content = substr($content, $remaining_bytes);
$part_ix ++;
and not $content = substr($content, $remaining_bytes+1); because of which a character gets missed

Rest all seems ok .. thanks a ton !!!:)
Post Reply