Page 1 of 2

Executing an application from PHP

Posted: Wed Jun 10, 2009 11:35 am
by Sekiryou
System:
Windows XP Pro
Service Pack 3 / .Net 3.5
Apache 2.2.11
PHP 5.2.9-2

Hi,
In a website project I have to convert mathematical formulas into .png images with LaTeX in my PHP code. The problem I have is that when I call LaTeX from PHP, the program takes for ever to execute and PHP crash after the timeout have been reached.
PHP Fatal error: Maximum execution time of 30 seconds exceeded in C:\WWW\lc.workspace.com\site\index.php on line 44
And this, even if everything went normally without any error with latex.exe. However, in command line, this process takes a fraction of a second to execute without any error too.

My homework so far:

Like suggested in another forum, I have added this to my php file:

Code: Select all

ini_set("max_execution_time", "240");
to "remove" the timeout error. It worked. But the process takes 2 minutes 47 secondes to execute. This takes too much time since latex.exe can be called several times in a single page to generate multiple mathematical formulas. I have tried thoses commands, exec(), shell_exec(), popen()/pclose(), all of them without any success. Here is some exemples I've tried:

Code: Select all

exec($command);
exec($command . " >NUL");
exec("start /B " . $command);
shell_exec($command);
pclose(popen("start ". $command, "r"));
 
$output = array();
$result = null;
$return = exec($command, $output, $result);
or
exec('start /B "'.$command.'"', $output, $return);
and many other I don't remember. It's always the same result and latex.exe takes 2 min 40 sec or so to execute from PHP.

Once the timeout problem resolved, I don't have any error in the Apache logs.

In php.ini, safe_mode is Off and disable_functions is Off like suggested elsewhere.

I have tried some simple commands to test for permissions, like "dir *.*" and "copy from to"... Every things works fine, with the dir command, the folder is listed in the output array. With the copy command, the file is copied and I have the message "1 file(s) copied." in the array. All of these tests with other command went fine and took a fraction of a second to execute.

I had the idea to look what was going on in the windows task manager to see what is happening. It's weird, when I execute latex.exe through PHP, the processor usage go to 90% to 100% during 2 minutes 40 secondes or so, until the process stop and then everything comes back to normal. When I do the same command in command line, I can't even see the process in the task manager, it's too fast. And CPU usage doesn't even budge. This isn't normal?

So here I am... I'm a bit lost now.

Thanks for your help and suggestions.
Sébastien.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 1:13 pm
by AlanG
Would it make any difference to run the script on the command line. You mentioned it for a website, so why not get let the script do it's job on the cli and then you can return the result to the browser. Essentially making web scripts and cli scripts interface. I wonder would that solve the problem? However, the drawback is that you would have to have PHP installed on the windows system separately.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 1:42 pm
by Sekiryou
AlanG wrote:[...]why not get let the script do it's job on the cli and then you can return the result to the browser.
I think it is impossible to do that because the website is a kind of wiki. If I would let the client do the convertion, they would need PHP, LaTeX, DVIPS, Imagick (convert and identify) to be installed on there system. This is unthinkable... It's a hell of a job to install those in a Windows environnement. I cannot have all of them install those software just to be able to use the website.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 2:18 pm
by AlanG
Sekiryou wrote:
AlanG wrote:[...]why not get let the script do it's job on the cli and then you can return the result to the browser.
I think it is impossible to do that because the website is a kind of wiki. If I would let the client do the convertion, they would need PHP, LaTeX, DVIPS, Imagick (convert and identify) to be installed on there system. This is unthinkable... It's a hell of a job to install those in a Windows environnement. I cannot have all of them install those software just to be able to use the website.
No my way of thinking left everything on the server. If I was messing with program interfaces the way your talking about, I would probably divide everything up into a more manageable way (such as dividing the website and the external processing into separate components.) Well, if the browser is too slow to achieve what you want, you will have to separate the processing from the browser. Maybe i'm wrong. I'd like to hear the solution when/if you find it.

Last minute idea:

Did you try using an output buffer? If you do, then the browser won't be receiving small amounts of information on a regular basis but will instead receive the lot in one lump sum. Could solve your problem.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 3:53 pm
by Sekiryou
AlanG wrote:If I was messing with program interfaces the way your talking about, I would probably divide everything up into a more manageable way (such as dividing the website and the external processing into separate components.)
It is already like that. I don't have any interface right now...
AlanG wrote:Well, if the browser is too slow to achieve what you want, you will have to separate the processing from the browser.
The browser have nothing to do with that... it's PHP... so server side. The client executes nothings.
AlanG wrote:Did you try using an output buffer?
All outputs are redirected. I have tried to redirect them to "nul", text file and variables but it's all the same.
AlanG wrote:If you do, then the browser won't be receiving small amounts of information on a regular basis but will instead receive the lot in one lump sum. Could solve your problem.
The browser don't receive anything... the script is executed on the server. What the browser receive is only the final product, the rendered HTML page in one lump sum like you say.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 4:29 pm
by AlanG
Sekiryou wrote:
AlanG wrote:If I was messing with program interfaces the way your talking about, I would probably divide everything up into a more manageable way (such as dividing the website and the external processing into separate components.)
It is already like that. I don't have any interface right now...
AlanG wrote:Well, if the browser is too slow to achieve what you want, you will have to separate the processing from the browser.
The browser have nothing to do with that... it's PHP... so server side. The client executes nothings.
AlanG wrote:Did you try using an output buffer?
All outputs are redirected. I have tried to redirect them to "nul", text file and variables but it's all the same.
AlanG wrote:If you do, then the browser won't be receiving small amounts of information on a regular basis but will instead receive the lot in one lump sum. Could solve your problem.
The browser don't receive anything... the script is executed on the server. What the browser receive is only the final product, the rendered HTML page in one lump sum like you say.
Right... first time I've ever had one of my posts dissected. Generally when I am helping users on forums, I tend to take shortcuts and bunch meanings together. I figure if I do this, the post seems less complicated for readers to understand. It's not often I fall into pitfalls where the OP is just out to test his own skills and prove how incorrect users who attempt to provide suggestions are. (Apologies if that is not what you are doing, but it sure seems that way.)

Anyway, in the context above, when I say "browser", I am generally referring to the actual browser, the client's computer and the remote user in general. I generally adjust my words depending on the context. If the topic was covering JavaScript, I would be sure to take that into account. As you are also dealing with command line scripts, I also added the web server/web interface into the mix. Works well when dealing with amateur users, but I do understand that it is too general when dealing with experienced developers.

Right, so jumping straight to the 2nd quote, What do you mean PHP can't be run in the browser?... DUH!!! I am referring to the web server here. If it is taking too long for the web server to process the information, then you will have to directly run a script placed on the system, as it might be faster.

As for the output buffer, you are wrong. The webserver passes information back to the client as soon as it is available. By using an output buffer, you can avoid the excessive overhead and transfer this information all at once. I was suggesting this, as if there was small bits of information being passed to the client, this could be the source of your problem.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 5:31 pm
by Sekiryou
AlanG wrote:Right... first time I've ever had one of my posts dissected. Generally when I am helping users on forums, I tend to take shortcuts and bunch meanings together. I figure if I do this, the post seems less complicated for readers to understand. It's not often I fall into pitfalls where the OP is just out to test his own skills and prove how incorrect users who attempt to provide suggestions are. (Apologies if that is not what you are doing, but it sure seems that way.)
It wasn't my goal, I assure you. English is not my native language... sorry about my interpretation of what you said. So nothing to worry about here :D
AlanG wrote:Anyway, in the context above, when I say "browser", I am generally referring to the actual browser, the client's computer and the remote user in general. [...] As you are also dealing with command line scripts, I also added the web server/web interface into the mix. [...] but I do understand that it is too general when dealing with experienced developers.
Ok, I understand what you mean when talking about interface now.
AlanG wrote:Right, so jumping straight to the 2nd quote, What do you mean PHP can't be run in the browser?... DUH!!! I am referring to the web server here. If it is taking too long for the web server to process the information, then you will have to directly run a script placed on the system, as it might be faster.
Isn't it what I'm doing when I use:

Code: Select all

$command = 'C:/LaTeX/MikeTeX/miktex/bin/latex.exe --interaction=nonstopmode --output-directory=C:/WWW/lc.workspace.com/site/tmp C:/WWW/lc.workspace.com/site/tmp/c0ec8631c0327e613311aa7f4a48e553.tex'
$output = array();
$result = null;
$return = exec($command, $output, $result);
AlanG wrote:As for the output buffer, you are wrong. The webserver passes information back to the client as soon as it is available. By using an output buffer, you can avoid the excessive overhead and transfer this information all at once. I was suggesting this, as if there was small bits of information being passed to the client, this could be the source of your problem.
For the exec() fonction, I thought that the output was written to an array that was filled during execution and when ending, it returns the array?
php.net Manual wrote:output
If the output argument is present, then the specified array will be filled with every line of output from the command. Trailing whitespace, such as \n, is not included in this array. Note that if the array already contains some elements, exec() will append to the end of the array. If you do not want the function to append elements, call unset() on the array before passing it to exec().
Anyway... all this is executed on the server, so nothing is sent to the network. So nothing should slow the process down like this?

Your reply made me test something, i've tried to execute the PHP script directly from the command line (C:\WWW\lc.workspace.com\site>php index.php) and guess what !? Everything goes fine and latex.exe is executed in a fraction of a second. Could it be Apache slowing the execution of the program??

Thanks for your help.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 8:06 pm
by AlanG
Apologies for the rant, but I thought it necessary. I guess it's my fault for assigning one name to a bunch of different things.

But yes, you are understanding what I am saying now. The web server could be slowing it down, so separate the components.

Web server - > < - Command Line

The command line script will operate independantly, and will return a value to the web server.
You will have to install PHP separately on the system aswell, away from apache.

You could then achieve it like so:

Code: Select all

$returned = shell_exec('php myscript.php');
Or something like that, I have never actually done it before so I am not too familiar with how it would actually work.

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 9:15 pm
by Sekiryou
AlanG wrote:You will have to install PHP separately on the system aswell, away from apache.
I'm not sure I understand what you mean here. Can you explain a bit please?

Re: Executing an application from PHP

Posted: Wed Jun 10, 2009 10:00 pm
by AlanG
Well PHP is installed as a module on your web server. In order to execute scripts separately from the web server you will need to install PHP onto the windows system separately.

Re: Executing an application from PHP

Posted: Tue Jun 16, 2009 1:16 pm
by Sekiryou
A little update here.

After many trials and fails, I've decided to make a little test environnement and installed EasyPHP 3.0. Just to see what would happen. Guess what, everything works perfectly. Good, so i'm on the right path.

I've started by comparing EasyPHP config files (httpd.conf and php.ini) with the original ones I was using with a file compare tool. The weird thing is that they are virtually identical (except for some variables EasyPHP use like the $path thing).

I've retested with the version of Apache and PHP that EasyPHP uses. Same problem occurs even with an identical php.ini and httpd.conf. So there must be a thing EasyPHP do that I don't !

Now, I know it's possible to make this to work because it work in EasyPHP. But I can't figure out what is different.

Does that help somebody to help me ?

Thanks
Sébastien

Re: Executing an application from PHP

Posted: Tue Jun 16, 2009 1:34 pm
by Eric!
I would just install standard php5. The suggestion is to use the command line php processor instead of the apache web module which has other overhead problems. Php5 shouldn't be hard to install. I've never heard of EasyPHP (but that isn't saying much).

Do these things need to generated dynamically? Can't you setup a cron job and a script to check/regenerate only if something is changed? Or just have the script that saves the edits reprocess the formulas for display. This will make your page run a lot faster.

Re: Executing an application from PHP

Posted: Tue Jun 16, 2009 1:59 pm
by Sekiryou
Eric! wrote:I would just install standard php5. The suggestion is to use the command line php processor instead of the apache web module which has other overhead problems. Php5 shouldn't be hard to install. I've never heard of EasyPHP (but that isn't saying much).
Like I said, EasyPHP was only for testing to see if the result would be the same than the standard PHP5 install. I've tried installing PHP as CGI too, that could solve my problem. But I'm unable to make it to work, I've tried many tutorials, none of them worked so I've stopped there (replacing one problem by another is not necessarely the best idea.). And I've also read about some security risks when using PHP as CGI.
Eric! wrote:Do these things need to generated dynamically? Can't you setup a cron job and a script to check/regenerate only if something is changed? Or just have the script that saves the edits reprocess the formulas for display. This will make your page run a lot faster.
Yes, they are generated dynamically and they are cached afterward for futur use. I don't know much about cron jobs... :?

Normally the process takes a fraction of a second... I just want my php script to execute latex.exe at the same speed it does when in command line.

Re: Executing an application from PHP

Posted: Tue Jun 16, 2009 3:37 pm
by Eric!
It seems doing this dynamically in the long run might be slow. I don't think windows will handle multiple exec() calls from multiple users very well.

Why not just regenerate them when there is a change and provide the user with a "saving..." animation while it struggles with exec(latex)? In the meantime you could experiment with fixing the problem, or just accept the latency while saving changes.

I found a http://bugs.php.net/bug.php?id=44994 bug report on the exec command, while it isn't exactly what you are seeing, someone recommended this work around for both exec and shell_exec. It seems you could also try to down grade to php5.1.x and try that as well.

Code: Select all

 
$WshShell = new COM("WScript.Shell");
$OutputFile = "C:\\Temp\\output.txt";              
$Command = "cmd /C MyApp.exe > $OutputFile";
$WshShell->Run( $Command, 0, true );
$data=file_get_contents( $OutputFile );  //you can fwrite this $data to wherever            
unlink( $OutputFile );
 
I can't really comment on the above workaround...it might work.

EDIT: fixed bug in file_get_contents line

Re: Executing an application from PHP

Posted: Wed Jun 17, 2009 12:49 pm
by Sekiryou
Eric! wrote:I found a http://bugs.php.net/bug.php?id=44994 bug report on the exec command, while it isn't exactly what you are seeing, someone recommended this work around for both exec and shell_exec. It seems you could also try to down grade to php5.1.x and try that as well.

Code: Select all

 
$WshShell = new COM("WScript.Shell");
$OutputFile = "C:\\Temp\\output.txt";              
$Command = "cmd /C MyApp.exe > $OutputFile";
$WshShell->Run( $Command, 0, true );
$data=file_get_contents( $OutputFile );  //you can fwrite this $data to wherever            
unlink( $OutputFile );
 
Yeah, I've seen a bunch of bug report similar to my problem. But it seems that PHP Team and Apache Team are throwing out the ball at each other. PHP Team say it's an Apache problem and Apache Team say it's a PHP problem!

Anyway, I've tried with the COM object but it is still the same result, the process just hang for more than 2 minutes.

Now, I was wondering what EasyPHP was doing that i'm not. Because with a 100% identical configuration files (PHP and Apache), the problem does not occurs using EasyPHP. I discovered that EasyPHP does not start Apache in Service Mode. So, I reinstalled PHP and Apache and I have made Apache start as a console application. And guess what ? The problem just vanished. The problem seems to be when Apache is started as a Windows Service. Now I have this annoying cmd.exe window in my task bar but at least, everything works according to my needs. I will probably reply on some PHP bug reports to tell them my findings. Could help someone else ! Who knows?

Thank all of you very much for your help. I think i've tried everything on that one !
Thanks again.

Sébastien.