Page 1 of 2

Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 11:58 am
by jwatson
I basically wrote a server image push in PhP which is basically the following:

Code: Select all

 
$sep = "sOmErAnDoMsTrInGfOrBoUnDaRy";
$header = 'Content-Type: multipart/x-mixed-replace;boundary=' . $sep;
header($header);
header("pragma: no-store,no-cache");
header("cache-control: no-cache,no-store,must-revalidate,max-age=-1");
header("expires: -1");
$boundary = "\n" . $sep . "\n";
echo $boundary;
set_time_limit(0);
while (1) {
    $handle = fopen("image.jpg", "rb");
    if ($handle != false) {
        $imageContents = "";
        while (!feof($handle)) {
            $imageContents .= fread($handle, 4096);
        }
        fclose($handle);
        echo "Content-Type: image/jpeg\n\n";
        echo $imageContents;
        echo $boundary;
        ob_flush();
        flush();
        usleep($waitTime);      
    }           
}       
 
However it seems to leak memory after a long period of time. Is there some way to avoid this? Do endless server push routines cause problems in PhP?

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 12:46 pm
by josh
Define leak memory, I'm not sure of the interal workings of apache but its probably apache holding hte document buffer in memory, PHP is executed as an extension of apache so anything PHP or the server itself puts in memory is going to show up owned by httpd, assuming that file never gets unlinked your loop is infinite so I'd assume that'd be quite one large buffer

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 1:11 pm
by jwatson
Yeah the httpd process (apache) is gaining memory over time. Is there any way to flush out this memory in each loop through the PhP? Maybe some kind of PhP.ini setting or something?

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 2:12 pm
by josh
theres implicit output buffer flushing, but the issue is that a flush is just changing the owner of that buffer from PHP to apache, the buffer is still sitting in the memory in the same spot so to speak, what I would do is spawn a background process since this seems like a batch task, then set up a queue database to keep track of tasks and when they finish, then you could have an ajax frontend or something of the like or simply just send the user an email when the task finishes, to a URL where they can view the outputs ( output would be saved for later user viewing ).

The web is only meant to do so much directly, sometimes you'll have to write "offline" code so to speak. Perhaps the problem is your data belongs on the disk and not in memory

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 4:59 pm
by jwatson
Nope I must have an endless server push to achieve what I want. Basically I have a camera generating an image on the server which I want to push out to the connected client.

I already have a polling version working, but I need to achieve a push version as well.

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 6:27 pm
by josh
HTTP is a stateless protocol. To implement a simulated persistent ( or asynchronous ) interaction you should use ajax, make one call at a time and then append the results to the end of the document

Anyways you can debug memory usage by calling http://us3.php.net/manual/en/function.m ... -usage.php before and after certain events and comparing how much the memory usage increases or decreases.


You could very easily replace an image on the page with the image that a script returns, call that script on a timer and achieve the effect you described, all without needless threads hanging on the server and the overhead of holding previous image data in memory

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 7:55 pm
by jwatson
I think we are missing the design concept of server push here. There is no request being made by the client except for the initial request only. The server spawns this PhP thread to endlessly stream image updates via content type multipart/x-mixed-replace telling the application to replace its content very quickly.

There is no browser here, no AJAX, nothing but one simple HTTP request coming from the client side. The server then handles the stream from this point forward.

Re: Image Server Push - Memory Leak?

Posted: Thu Oct 02, 2008 9:17 pm
by josh
Well I understand the concept just fine, I'm just saying its not what http is made for. There are other protocools / design approaches to what you're trying to do. Anyways. You dont have output buffering enabled in php.ini by any chance do you? Try disabling that, and play around with those memory functions... I think theres some information thats being overlooked here, what you're trying to do should be possible without using up all that memory from a technical standpoint

Re: Image Server Push - Memory Leak?

Posted: Fri Oct 03, 2008 3:19 pm
by jwatson
I know I am kind of forced into this design by someone's bad code. Originally I wanted to use the AJAX approach and it was beautifully working until this point.

Turn off output buffering eh? I'll try it, at this point I am suffering an apache module written in C++ to attempt the same thing. Thanks for your time though, I really appreciate the insight, anyone accomplish an endless server push with PhP?

Re: Image Server Push - Memory Leak?

Posted: Mon Oct 06, 2008 10:53 am
by jwatson
Turning off output buffering does NOT work.

Re: Image Server Push - Memory Leak?

Posted: Mon Oct 06, 2008 7:26 pm
by josh
ob_flush:
This function does not destroy the output buffer like ob_end_flush() does.
Also if the file does not exist that code would run an infinite loop

Re: Image Server Push - Memory Leak?

Posted: Tue Oct 07, 2008 5:14 pm
by jwatson
No go with ob_end_flush(), the file always exists. Apache seems to be allocating 8K every now and then, for what reason I don't know. Is there a better way to achieve an image push?

Re: Image Server Push - Memory Leak?

Posted: Tue Oct 07, 2008 6:01 pm
by josh
well you need to isolate which code is allocating the memory, (using the memory debugging functions I told you about )

For instance if you stop reading files and stop sending output does it keep using memory? And if the file always exists why bother checking if $handle == false

Re: Image Server Push - Memory Leak?

Posted: Tue Oct 07, 2008 6:39 pm
by VladSun
jshpro2 wrote:And if the file always exists why bother checking if $handle == false
It's even worse if it doesn't exist, beacuse no sleep() will be executed...

Re: Image Server Push - Memory Leak?

Posted: Tue Oct 07, 2008 7:06 pm
by josh
Also of interest:
If a globalized variable is unset() inside of a function, only the local variable is destroyed. The variable in the calling environment will retain the same value as before unset() was called.
Basically theres too many different things that could be causing it to know for sure without using those memory functions.