Parsing common log files...

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
kmitchel46787
Forum Newbie
Posts: 3
Joined: Fri Feb 21, 2003 2:41 pm

Parsing common log files...

Post by kmitchel46787 »

I'm rather new to php, and need some pointers...

Working on a script to parse a common log file (same format used by Apache) and insert the data into a MySql db. Another series of scripts will query the db to look for users, etc. Any suggestions on how to parse a log entry into it's components? Thanx.
kmitchel46787
Forum Newbie
Posts: 3
Joined: Fri Feb 21, 2003 2:41 pm

Post by kmitchel46787 »

Anyhelp?
User avatar
BDKR
DevNet Resident
Posts: 1207
Joined: Sat Jun 08, 2002 1:24 pm
Location: Florida
Contact:

Post by BDKR »

I like to use file() to open a log file. I've used it with files greater than 40 megs. Anyways, as it creates an array, just iterate over that array and parse the individual elements. Now while parsing, you'll need to break the line up. Explode() is good for that. If you know the format of the log file, and that format remains constant, then you shouldn't have any problems.

An example....

Code: Select all

$log_array=file("/var/log/dolt.log");
$x=sizeof($log_array);
$y=o;
while($y<=$x)
    {
    $info_array=explode("some_delimiter", $log_array[$y]);
    /* Do stuff with elements of the info_array here */
    }
Cheers,
BDKR
kmitchel46787
Forum Newbie
Posts: 3
Joined: Fri Feb 21, 2003 2:41 pm

Post by kmitchel46787 »

I ended up doing multiple splits to accomidate different dilemiters. Having problems removing duplicate records with MySQL, and no subselects. I had my import script look for an existing record before commiting it to the DB. This worked allright until there were more than a million records. Looks like I'll try postgres instead.
Post Reply