PHP Job Queue?

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
Garett_Haight
Forum Newbie
Posts: 3
Joined: Fri Sep 03, 2010 2:24 pm

PHP Job Queue?

Post by Garett_Haight »

Hello all,

I'm working on a script that allows users to upload photos to this project's website that automatically
resizes, crops and generates galleries for the photos. I'm using the GD library to manipulate the images
and since they're usually large jpeg files straight from the user's camera, the file processing takes several
minutes.
What I would like to do, is to have a setup which allows the users to upload the photos, then photos would
be added to a queue and processed on the server, allowing the user to navigate away from the page. I would
also like to avoid having more than one image processed at a time since the large images take up a significant
amount of server memory.
So.. I suppose my question would be: Is there a way to have a php file execute on the server whenever there are
items listed in the queue table?

Thanks!
cpetercarter
Forum Contributor
Posts: 474
Joined: Sat Jul 25, 2009 2:00 am

Re: PHP Job Queue?

Post by cpetercarter »

A partial answer to your problem would be to use a pop-up window for image upload and processing. The user could then navigate to other parts of the site from the main widow while the image processing was going on in the popup.
User avatar
Jonah Bron
DevNet Master
Posts: 2764
Joined: Thu Mar 15, 2007 6:28 pm
Location: Redding, California

Re: PHP Job Queue?

Post by Jonah Bron »

I don't know for sure, but I think you could run a PHP script in the background by running

Code: Select all

shell_exec('php "somephpscript.php" &' . PHP_EOL . PHP_EOL);
Assuming you're server runs Linux, this will start PHP in the background running somephpscript.php. I'm not sure how the relative url would be set up, so it would be best to use an absolute url.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: PHP Job Queue?

Post by Weirdan »

The easiest way would be accept the file, store a record in the database and then have a script running from cron pick it up and convert the file. You can also look into Gearman - it's kind of distributed message bus with support for both async and sync communications between client and worker(s).

However, the more interesting and scalable solution would be to make user's browser to do all the grunt work for you - it's certainly possible to perform most of the graphic operations with flash and upload ready to use images. Using javascript + canvas could also be an option - however I'm not sure if you can load local image into canvas (if not it should be possible to make flash read the file and yield data to javascript).
User avatar
McInfo
DevNet Resident
Posts: 1532
Joined: Wed Apr 01, 2009 1:31 pm

Re: PHP Job Queue?

Post by McInfo »

I second Weirdan's motion to enlist the user's browser to handle the image scaling. Maybe Java applets would be worth investigating.
User avatar
yacahuma
Forum Regular
Posts: 870
Joined: Sun Jul 01, 2007 7:11 am

Re: PHP Job Queue?

Post by yacahuma »

I am using gearman + supervisord. It worked great so far.
eFishy
Forum Newbie
Posts: 9
Joined: Tue Jan 05, 2010 12:26 pm

Re: PHP Job Queue?

Post by eFishy »

Personally I would save the uploaded file on the server and keep a record in mySQL (or what ever database that your using).
Then run a cron every minute that looks for images in the que and process them as required.
Alternatively you could use folders and list their contents for the queuing system if you don't want to use a database.
User avatar
yacahuma
Forum Regular
Posts: 870
Joined: Sun Jul 01, 2007 7:11 am

Re: PHP Job Queue?

Post by yacahuma »

Thecool thing about gearman is that you dont have to wait that minute for the cron. It is instant
User avatar
pickle
Briney Mod
Posts: 6445
Joined: Mon Jan 19, 2004 6:11 pm
Location: 53.01N x 112.48W
Contact:

Re: PHP Job Queue?

Post by pickle »

Also, look into imagemagick instead of GD. I've heard GD is quite a bit slower & uses more memory. Imagemagick you can invoke right from the command line, so you could use just a shell script rather than a full PHP script - should save you some clock cycles & memory usage.
Real programmers don't comment their code. If it was hard to write, it should be hard to understand.
Garett_Haight
Forum Newbie
Posts: 3
Joined: Fri Sep 03, 2010 2:24 pm

Re: PHP Job Queue?

Post by Garett_Haight »

Thanks for the suggestions everyone! I have some research to do now :)

I'd love to use imagemagick, but I'm working with this Synology server that
doesn't come with it and it seems that I'd have to do a lot to get it installed.
I've had to do a lot of working around the limitations of this server...
Garett_Haight
Forum Newbie
Posts: 3
Joined: Fri Sep 03, 2010 2:24 pm

Re: PHP Job Queue?

Post by Garett_Haight »

Actually... it looks like there's something about imagemagick on here....
it's not listed in the php info, but I found an application on the server
that has imagemagick functions....
Post Reply