I want to run a cron job of a php script. The script loops/curls ~81,000 webpages so it takes about 5-8 hours to complete (i haven't had enough time to time it!).
I've read on other forums that cron has a limited php runtime of 15 minutes for Ubuntu dapper drake. I was curious as to lengthen the runtime of php scripts.
Why not break up the number of pages into blocks of say 100 or ten? You can then run the script more often and have it check which blocks aren't in use taking the first one for itself.
I think you use MySQL's "OFFSET" and "LIMIT" commands - it should be fairly straightforward. In fact, as long as you sort by last_update_time you can get away with just using "LIMIT".