I have a script that reads in data from a flat text file. What it does is it chops the file up into chunks based on the file-size that it is given. For instance, if i set the file-size limit to 80k, it is going to split the file when it reaches 80k, and will continue to do so until it reaches the end of the flat text file.
It looks something like this:
- "rawtextfile.txt" (goes into script ...)
and what comes out of it looks like this:
- "rawtextfile.txt.chunk1" (80k)
- "rawtextfile.txt.chunk2" (80k)
- "rawtextfile.txt.chunk3" (whatever bytes left over ...)
What i want it to do is i want it to split the file as soon as it reaches the end of the line rather than splitting it exactly at 80k. I ask this because it is chopping right through a line and continuing that line in the next file. I want it to reach the end of the line first before splitting the file.
Is that possible?
Here is what i have so far:
Code: Select all
function file_chop_big($file_path, $chunk_size) {
$size = filesize($file_path);
//find number of full $chunk_size byte portions
$num_chunks = floor($size / $chunk_size);
$file_handle = fopen($file_path, 'r');
$chunks = array();
for($kk = 0; $kk < $num_chunks; $kk++) {
$chunks[$kk] = basename($file_path).'.chunk'.($kk + 1);
$chunk_handle = fopen($chunks[$kk], 'w'); //open the chunk file for writing
//write the data to the chunk file 1k at a time
while((ftell($chunk_handle) + 1024) <= $chunk_size) {
fwrite($chunk_handle, fread($file_handle, 1024));
} // end while-loop.
if(($leftover = $chunk_size - ftell($chunk_handle)) > 0 ) {
fwrite($chunk_handle, fread($file_handle, $leftover));
} // end if.
fclose($chunk_handle);
} // end for-loop.
if(($leftover = $size - ftell($file_handle)) > 0) {
$chunks[$num_chunks] = basename($file_path).'.chunk'.($num_chunks + 1);
$chunk_handle = fopen($chunks[$num_chunks], 'w');
while(!feof($file_handle)) {
fwrite($chunk_handle, fread($file_handle, 1024));
} // end while-loop.
fclose($chunk_handle);
} // end if.
fclose($file_handle);
return $chunks;
} // end function.
Thanks