Page 1 of 1
how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 12:17 am
by ummar1
i am downloading some log file (in which 100 lines are logged per sec) from ftp server via php.
every time i download the file download becomes slower as it size grows with time.
is there some way that i get only the new lines (added later) so that my download becomes faster.
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 6:51 am
by litebearer
8.6 million lines per day? Perhaps you should re-think how the log file is created/modified.
Maybe use datetime as the logfile name - ie 2011031000 would be from midnight til 00:59:59 ETC ETC
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 9:46 am
by pickle
Rotate your log - either on a time basis or file size basis.
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 10:43 am
by ummar1
thx for ur reply dear but i cannot alter my log file

, i just have to download the file.
this file is generated by some other member and he just keep on doing his work and i have to do mine
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 10:49 am
by pickle
Then there's not really anything you can do. You should really tell the other member to rotate his logs though.
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 12:03 pm
by litebearer
How frequently do you need to download the log file?
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 9:43 pm
by ummar1
every sec
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 10:01 pm
by litebearer
seeing how it takes longer than a sec to download the log file, it seems you have multiple downloads going simultaneously?
Re: how to downlaod only new lines of a file via ftp
Posted: Thu Mar 10, 2011 10:13 pm
by jim.barrett
so you have a file that is growing at a rate of 100 lines per second and you need to upload it every second?? I really hope someone has an answer for this because I would love to hear it.
Re: how to downlaod only new lines of a file via ftp
Posted: Sat Mar 12, 2011 7:56 am
by ummar1
oh i have a simple solution

, i have written a simple c shell script
#!/bin/csh
while
do
tale -100 log.txt > downloadandandappend.txt
done
what this simple test do is it reads the last 100 lines of my file "log.txt" file (in linux) and then send it to a newly created file "downloadandandappend.txt " and then keeps on doing so
then i download this "downloadandandappend.txt " which always recreates on every cycle of while statement.
so i have the solution but for this i have to run the script on linux side first, more efficient solution is that if i can run this shell script from php. i know the function "shell_exec" but it says that i cannot run this, php is not in safe mode.
how can i run this shell script command help me please.
i am using php designer version 7.2.2 and wamp server