Is it possible to run a CLI PHP script with unlimited memory from PHP perspective but restrict how much processor/memory it consume at the system level?
The reason I ask, is because I have a script which needs to spider a massive HTML file (over 130 MB). The spidering program doesn't seem to work when I partition the massive file into smaller chunks and support isn't not responding to my request.
As a last ditch attempt I figure I can maybe set the memory to unlimited in PHP/MySQL but some how control the RAM the processes get and hopefully virtual memory will do the trick.
Cheers,
Alex
CLI script memory use
Moderator: General Moderators
-
- DevNet Evangelist
- Posts: 6267
- Joined: Tue Dec 21, 2004 5:00 pm
- Location: Winnipeg
Re: CLI script memory use
ulimit -m
cpulimit
cpulimit
-
- DevNet Evangelist
- Posts: 6267
- Joined: Tue Dec 21, 2004 5:00 pm
- Location: Winnipeg
Re: CLI script memory use
Thanks you for that.
So from quickly reading the docs, I should login as a regular user, set the CPU limit with the-m option to limit memory consumption? Do I need to call these or added these in a bash script or can I call them in a one off setting (I don't really need them after I run this spider program)?
Secondly, while the php.ini for CLI is set to unlimited memory use (script was crashing due to memory shortage which setting to -1 fixed) will limiting the system memory cause the same affect or will it just take longer?
thanks Weirdan
Cheers,
Alex
So from quickly reading the docs, I should login as a regular user, set the CPU limit with the-m option to limit memory consumption? Do I need to call these or added these in a bash script or can I call them in a one off setting (I don't really need them after I run this spider program)?
Secondly, while the php.ini for CLI is set to unlimited memory use (script was crashing due to memory shortage which setting to -1 fixed) will limiting the system memory cause the same affect or will it just take longer?
thanks Weirdan
Cheers,
Alex
Re: CLI script memory use
as for ulimit - you just set limits, then run your program:
ulimit -m limits resident memory size for a program, so theoretically when the program exceeds this limit it should just start swapping.
as for cpulimit - I've never used it, so you're on your own here.
Code: Select all
~$ ulimit -m $((1024*1024))
~$ nohup php path/to/script.php &
~$ ulimit -m unlimited
as for cpulimit - I've never used it, so you're on your own here.