Page 1 of 1

Writing to a file till a size is reached

Posted: Thu Oct 09, 2008 12:36 pm
by kkonline
Hi,
I am writing to a file using fopen and other commands to keep writing some contents to that file UNTIL A PARTICULAR SIZE ex 40KB is reached.

The max size of the file should be around 40kb; how can I perform this task

I wrote

Code: Select all

 
$snoopy = new Snoopy;
 
$myFile = "testfile.txt";
echo (filesize($myFile));   
while (filesize($myFile)<40000){
echo "file opened";
$fh = fopen($myFile, 'a') or die("can't open file");
 
    $url="kkonline.org";
$snoopy->fetchtext($url);
$resulttext= $snoopy->results;
 
$stringData = $resulttext;
fwrite($fh, $stringData);
$stringData = "\n";
fwrite($fh, $stringData);
fclose($fh);
echo $resulttext;
}
 
But not getting the desired results.

Re: Writing to a file till a size is reached

Posted: Thu Oct 09, 2008 4:00 pm
by VladSun
Weird ...
It seems that fclose() doesn't flush the file indeed... It's flushed after the script has finished its execution. Nasty :)

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 8:36 am
by kkonline
Got it working

Code: Select all

$fh = fopen($myFile, 'a') or die("can't open file");
 
// needed otherwise filesize() may return wrong size
clearstatcache();
 
// calc how many bytes we can write
$remaining_bytes = 40000 - filesize($myFile);
 
// truncate our data if its too big
if (strlen($resulttext) > $remaining_bytes) {
    $resulttext = substr($resulttext, 0, $remaining_bytes);
}
 
fwrite($fh, $resulttext); 
 
I am thinking of creating files dynamically; writing 40KB of data in each and the remaining data on the resulttext to "myfile".$i.".txt"

such that myfile1.txt with 40KB data then the data will continue on myfile2.txt and on reaching 40KB limit will be written on myfile3.txt.

Is this possible and how can I implement this; any idea?

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 8:48 am
by VladSun
Heh, I can't belive I missed the "Notice" section in filesize() man page :)

A quick question - what's the main idea of having these multiple files? What's the reason?

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 9:01 am
by kkonline
Basically I am developing a simple logic for displaying the feeds; however the file grows at a very fast rate and because of which it doesn't get indexed (due to big file; loading time increases) so I am creating a number of files each with say 40kb of data only.

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 9:34 am
by VladSun
untested!

Code: Select all

// return the PART_IX of the last file used
function write_feed($file_name, $content, $part_ix = 0, $max_length = 4000)
{
    clearstatcache();
    
    $finished = false;
    
    while (!$finished)
    {
        $full_file_name = $file_name.($part_ix == 0 ? '.txt' : '.'.$part_ix.'.txt');
        $handle = @fopen($full_file_name, 'a+') or die("can't open file ".$full_file_name);
 
        $remaining_bytes = $max_length - filesize($full_file_name);
        if (strlen($content) > $remaining_bytes)
        { 
            $partial_content = substr($content, 0, $remaining_bytes);
            $content = substr($content, $remaining_bytes + 1);
            $part_ix ++;
        }
        else
        {
            $partial_content = $content;
            $finished = true;
        }
         
        fwrite($handle, $partial_content);
        fclose($handle);
    }
    return $part_ix;
}

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 10:50 am
by kkonline
Hi,

Code: Select all

$snoopy = new Snoopy;
 
$snoopy->fetchtext($url);
$resulttext= $snoopy->results;
 
$myFile = "testfile.txt";
 
echo "file opened";
$fh = fopen($myFile, 'a') or die("can't open file");
 
// needed otherwise filesize() may return wrong size
 
clearstatcache();
 
// calc how many bytes we can write
 
$remaining_bytes = 5000 - filesize($myFile);
 
// truncate our data if its too big
 
if (strlen($resulttext) > $remaining_bytes) {
 
    $resulttext = substr($resulttext, 0, $remaining_bytes);
 
}
 
//fwrite($fh, $resulttext); 
 
//////////////////////////
 
//search and replace
$patterns[0] = '/"/';
$patterns[1] = '/Comments/';
$patterns[2] = '/Have something you would like to share?/';
$replacements[2] = '<p>';
$replacements[1] = '';
$replacements[0] = '';
$replaced=preg_replace($patterns, $replacements, $resulttext);
//search and replace
 
$stringData = $replaced;
fwrite($fh, $stringData);
$stringData = "\n";
fwrite($fh, $stringData);
fclose($fh);
//echo $replaced;
 
////////////////
 // return the PART_IX of the last file used
 function write_feed($file_name, $content, $part_ix = 0, $max_length = 4000)
 {
     clearstatcache();
    
     $finished = false;
    
     while (!$finished)
     {
         $file_name .= ($part_ix == 0 ? '.txt' : '.'.$part_ix.'.txt');
         $handle = @fopen($file_name, 'a+') or die("can't open file ".$file_name);
 
         $remaining_bytes = $max_length - filesize($file_name);
         if (strlen($content) > $remaining_bytes)
         {
             $partial_content = substr($contents, 0, $remaining_bytes);
             $content = substr($contents, $remaining_bytes + 1);
             $part_ix ++;
         }
         else
            $finished = true;
        
          
         fwrite($handle, $content);
         fclose($handle);
     }
     return $part_ix;
}
 
$last=write_feed($myFile,$replaced,0,4000);
echo $last;
The code above ;
1> The files are created in this fashion testfile.txt then testfile.txt.txt then testfile.txt.txt.1.txt
2> The output is

Code: Select all

Quotes
 
file opened
Notice: Undefined variable: contents in c:\program files\easyphp\www\snoopy\index_v2.php on line 87
 
Notice: Undefined variable: contents in c:\program files\easyphp\www\snoopy\index_v2.php on line 88
1
3> Only the forst file testfile.txt has contents rest all are blank

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 11:08 am
by VladSun
:) You've tested it, I've edited it :)

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 11:15 am
by kkonline
Code I tested is

Code: Select all

$snoopy = new Snoopy;
 
$snoopy->fetchtext($url);
$resulttext= $snoopy->results;
 
$myFile = "testfile";
 
echo "file opened";
$fh = fopen($myFile, 'a') or die("can't open file");
 
// needed otherwise filesize() may return wrong size
 
clearstatcache();
 
// calc how many bytes we can write
 
$remaining_bytes = 5000 - filesize($myFile);
 
// truncate our data if its too big
 
if (strlen($resulttext) > $remaining_bytes) {
 
    $resulttext = substr($resulttext, 0, $remaining_bytes);
 
}
 
//fwrite($fh, $resulttext); 
 
//////////////////////////
 
//search and replace
$patterns[0] = '/"/';
$patterns[1] = '/Comments/';
$patterns[2] = '/Have something you would like to share?/';
$replacements[2] = '<p>';
$replacements[1] = '';
$replacements[0] = '';
$replaced=preg_replace($patterns, $replacements, $resulttext);
//search and replace
 
$stringData = $replaced;
fwrite($fh, $stringData);
$stringData = "\n";
fwrite($fh, $stringData);
fclose($fh);
//echo $replaced;
 
////////////////
   // return the PART_IX of the last file used
    function write_feed($file_name, $content, $part_ix = 0, $max_length = 2000)
    {
        clearstatcache();
      
        $finished = false;
       
        while (!$finished)
        {
           $full_file_name = $file_name.($part_ix == 0 ? '.txt' : '.'.$part_ix.'.txt');
           $handle = @fopen($full_file_name, 'a+') or die("can't open file ".$full_file_name);
    
           $remaining_bytes = $max_length - filesize($full_file_name);
           if (strlen($content) > $remaining_bytes)
           {
              $partial_content = substr($content, 0, $remaining_bytes);
               $content = substr($content, $remaining_bytes + 1);
              $part_ix ++;
         }
         else
              $finished = true;
         
           
          fwrite($handle, $content);
           fclose($handle);
       }
       return $part_ix;
   }
 
$last=write_feed($myFile,$replaced,0,4000);
echo $last;
I tested this
1> The first file has contents of 4 kb ...ok
2> the second file has the contents which are repeated from the contents of 1st file
3>the third file also has the contents which are repeated from the contents of 1st file.

means 2nd and 3rd file have duplicate contents which are indeed repeated contents from 1st fileHello again,

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 11:24 am
by VladSun
Test

Code: Select all

$s = '';
for ($i=0; $i < 20; $i++)
    $s .= str_repeat($i, 200);
    
echo strlen($s);
 
write_feed('test', $s, 0, 200);
And try to fix errors, if there are some ;)

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 11:36 am
by kkonline
VladSun wrote:Test

Code: Select all

$s = '';
for ($i=0; $i < 20; $i++)
    $s .= str_repeat($i, 200);
    
echo strlen($s);
 
write_feed('test', $s, 0, 200);
And try to fix errors, if there are some ;)
The first file has 1 to 19
the second file has 2 to 19
the third has 3 to 19
and so on...


any suggestions

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 11:38 am
by VladSun
Yes - one.
Use my corrected code, not the second-edit one ;)

Re: Writing to a file till a size is reached

Posted: Fri Oct 10, 2008 11:59 am
by kkonline
Just one small change

Code: Select all

$partial_content = substr($content, 0, $remaining_bytes);
$content = substr($content, $remaining_bytes);
$part_ix ++;
and not $content = substr($content, $remaining_bytes+1); because of which a character gets missed

Rest all seems ok .. thanks a ton !!!:)