Loading text from an uploaded text file.

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
rellison
Forum Newbie
Posts: 7
Joined: Sat Dec 11, 2010 5:19 pm

Loading text from an uploaded text file.

Post by rellison »

Forgive if this has been asked I searched but could not find what i was looking for.

I have a database routine where a user will upload a text file that contains data. Each record of data in the text file ends with <EOR>
Each record can contain a random length of text possibly up to 3000 characters. And can have multiple CR's located in it.
I need to parse out the record up to and including the <EOR>. So once the file is read in up to the end of first <EOR> that string of data would be sent to the parse routines. Then the next record gets read in and so on till the end of the file.

I am using this a to do the routine now but the encountered CR's keep breaking the string to soon and not all of the record is getting read in...

Code: Select all

    		// Read the first line
    		$string = fgets ($file, 1024);
             	$string = strtoupper ($string);
                if (stristr($string, "<EOR>"))
        		{
        			// Process ADIF file
                               processFile ($file,$count,$duplicate,$Bad,$string,$usertable);
        		}
        		else
        		{
        			while (($string = fgets ($file,1024)) && !$valid_file)
        			{
        				$string = strtoupper ($string);
                                        if (stristr($string, "<EOR>") )
        			         {
					   $valid_file = 1;
                                           processFile ($file,$count,$duplicate,$Bad,$string,$usertable);
        				}
        			}
        
        			// No records in file found - exit with an error
        			if (!$valid_file)
        			{
        				echo "<P>Error - Unable to upload file: $browser_name\n";
        				echo "<P>Invalid file format.\n";
         				die();
        			}
        		}

Can someone show me the correct routine I should be using to do this.. This is driving me nuts since I am not that good with PHP yet..

Thanks Rick
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Loading text from an uploaded text file.

Post by Celauran »

rellison wrote:the encountered CR's keep breaking the string to soon and not all of the record is getting read in...
fgets():
Reading ends when length - 1 bytes have been read, on a newline (which is included in the return value), or on EOF (whichever comes first). If no length is specified, it will keep reading from the stream until it reaches the end of the line.
Why not use file_get_contents() and explode()?
rellison
Forum Newbie
Posts: 7
Joined: Sat Dec 11, 2010 5:19 pm

Re: Loading text from an uploaded text file.

Post by rellison »

That might work but my worry is some of the text files being uploaded might contain 100,000records or more. and the memory requirements on the server..
rellison
Forum Newbie
Posts: 7
Joined: Sat Dec 11, 2010 5:19 pm

Re: Loading text from an uploaded text file.

Post by rellison »

I just wanted to Bump this as I have still not figured a good way to do this.Reading in the entire file and then exploding the contents really would not work with the large files that get uploaded. MY thinking is It should read in up to the <EOR> and include that into the string that is read in I just do not know how to code that..

TIA
Rick
User avatar
McInfo
DevNet Resident
Posts: 1532
Joined: Wed Apr 01, 2009 1:31 pm

Re: Loading text from an uploaded text file.

Post by McInfo »

This is a generic enough problem that I decided to share my solution. The example displays its own source code.

Code: Select all

<?php
function main () {
    header('Content-Type: text/plain; charset=UTF-8');
    $file = fopen(basename(__FILE__), 'rb');
    chunkFile($file, 'echoRecord');
    fclose($file);
}
function echoRecord ($record) {
    echo $record, PHP_EOL;
}
function chunkFile ($handle, $callback, $delimiter=PHP_EOL, $chunkSize=1024) {
    $delimSize = strlen($delimiter);
    $lapSize = ($delimSize - 1) ?: 1;
    $chunk = '';
    $pos = 0;
    $record = '';
    while (!feof($handle)) {
        $chunk .= fread($handle, $chunkSize);
        $pos = strpos($chunk, $delimiter);
        if ($pos === false) {
            $record .= substr($chunk, 0, -$lapSize);
            $chunk = substr($chunk, -$lapSize);
        } else {
            $record .= substr($chunk, 0, $pos);
            $callback($record);
            $chunk = substr($chunk, $pos + $delimSize);
            $record = '';
        }
    }
    while ($chunk !== false) {
        $pos = strpos($chunk, $delimiter);
        if ($pos === false) {
            $callback($chunk);
            $chunk = false;
        } else {
            $callback(substr($chunk, 0, $pos));
            $chunk = substr($chunk, $pos + $delimSize);
        }
    }
}
main();
Let me know if you want it commented or if you find a problem.
Post Reply