parsing log for a specific condition and removing duplicates
Posted: Sat Feb 18, 2006 2:04 am
feyd | Please use
So far I've got to the point where I can grab all lines from the log file which contain http word...
I'd appreciate any suggestion!
feyd | Please use
Code: Select all
andCode: Select all
tags where appropriate when posting code. Read: [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url][/color]
Hi,
I'm new here and also just started learning PHP and already have a big problem (at least it's big for me )
Please help.
What I need is to parse the log file for a specific condition: I need to get all URLs from the log that are in "" (for example: [b]http://forum.novd.ru/index.php?showtopic=56816&st=15[/b])
and remove duplicates.
Here are 3 sample records from the access log (2 an 3 lines contain same URL so I need only one then):Code: Select all
213.148.171.244 - - [15/Feb/2006:22:49:52 -0800] "GET /index.jsp HTTP/1.1" 200 98238 "http://forum.novd.ru/index.php?showtopic=56816&st=15" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; ru) Opera 8.50"
68.105.89.121 - - [15/Feb/2006:22:50:05 -0800] "GET /index.jsp HTTP/1.1" 200 98161 "http://forums.us.comp.com/supportforums/board/message?board.id=dim_upghw&message.id=78270" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1"
68.105.89.121 - - [15/Feb/2006:22:50:07 -0800] "GET /index.jsp HTTP/1.1" 200 98161 "http://forums.us.comp.com/supportforums/board/message?board.id=dim_upghw&message.id=78270" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1"So far I've got to the point where I can grab all lines from the log file which contain http word...
Code: Select all
<?
$in="/usr/local/apache/domlogs/my_log.log";
$out = "final_log.txt";
$fp_in = fopen($in, 'r') or die("No such file!\n");
$fp_out = fopen($out, 'w') or die("Cannot open a new file!\n");
while (!feof($fp_in)) {
$data = fgets($fp_in, 1024);
if ($data == false) {
break;
}
if (preg_match ("/http/", $data)) {
fwrite($fp_out, $data) or die("Cannot write!\n");
}
}
fclose($fp_out);
fclose($fp_in);
?>feyd | Please use
Code: Select all
andCode: Select all
tags where appropriate when posting code. Read: [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url][/color]