how to downlaod only new lines of a file via ftp
Moderator: General Moderators
how to downlaod only new lines of a file via ftp
i am downloading some log file (in which 100 lines are logged per sec) from ftp server via php.
every time i download the file download becomes slower as it size grows with time.
is there some way that i get only the new lines (added later) so that my download becomes faster.
every time i download the file download becomes slower as it size grows with time.
is there some way that i get only the new lines (added later) so that my download becomes faster.
-
litebearer
- Forum Contributor
- Posts: 194
- Joined: Sat Mar 27, 2004 5:54 am
Re: how to downlaod only new lines of a file via ftp
8.6 million lines per day? Perhaps you should re-think how the log file is created/modified.
Maybe use datetime as the logfile name - ie 2011031000 would be from midnight til 00:59:59 ETC ETC
Maybe use datetime as the logfile name - ie 2011031000 would be from midnight til 00:59:59 ETC ETC
Re: how to downlaod only new lines of a file via ftp
Rotate your log - either on a time basis or file size basis.
Real programmers don't comment their code. If it was hard to write, it should be hard to understand.
Re: how to downlaod only new lines of a file via ftp
thx for ur reply dear but i cannot alter my log file
, i just have to download the file.
this file is generated by some other member and he just keep on doing his work and i have to do mine
this file is generated by some other member and he just keep on doing his work and i have to do mine
Re: how to downlaod only new lines of a file via ftp
Then there's not really anything you can do. You should really tell the other member to rotate his logs though.
Real programmers don't comment their code. If it was hard to write, it should be hard to understand.
-
litebearer
- Forum Contributor
- Posts: 194
- Joined: Sat Mar 27, 2004 5:54 am
Re: how to downlaod only new lines of a file via ftp
How frequently do you need to download the log file?
-
litebearer
- Forum Contributor
- Posts: 194
- Joined: Sat Mar 27, 2004 5:54 am
Re: how to downlaod only new lines of a file via ftp
seeing how it takes longer than a sec to download the log file, it seems you have multiple downloads going simultaneously?
-
jim.barrett
- Forum Newbie
- Posts: 20
- Joined: Tue Mar 01, 2011 5:47 am
Re: how to downlaod only new lines of a file via ftp
so you have a file that is growing at a rate of 100 lines per second and you need to upload it every second?? I really hope someone has an answer for this because I would love to hear it.
Re: how to downlaod only new lines of a file via ftp
oh i have a simple solution
, i have written a simple c shell script
#!/bin/csh
while
do
tale -100 log.txt > downloadandandappend.txt
done
what this simple test do is it reads the last 100 lines of my file "log.txt" file (in linux) and then send it to a newly created file "downloadandandappend.txt " and then keeps on doing so
then i download this "downloadandandappend.txt " which always recreates on every cycle of while statement.
so i have the solution but for this i have to run the script on linux side first, more efficient solution is that if i can run this shell script from php. i know the function "shell_exec" but it says that i cannot run this, php is not in safe mode.
how can i run this shell script command help me please.
i am using php designer version 7.2.2 and wamp server
#!/bin/csh
while
do
tale -100 log.txt > downloadandandappend.txt
done
what this simple test do is it reads the last 100 lines of my file "log.txt" file (in linux) and then send it to a newly created file "downloadandandappend.txt " and then keeps on doing so
then i download this "downloadandandappend.txt " which always recreates on every cycle of while statement.
so i have the solution but for this i have to run the script on linux side first, more efficient solution is that if i can run this shell script from php. i know the function "shell_exec" but it says that i cannot run this, php is not in safe mode.
how can i run this shell script command help me please.
i am using php designer version 7.2.2 and wamp server