CLI script memory use

[alex.barylski]

Is it possible to run a CLI PHP script with unlimited memory from PHP perspective but restrict how much processor/memory it consume at the system level?

The reason I ask, is because I have a script which needs to spider a massive HTML file (over 130 MB). The spidering program doesn't seem to work when I partition the massive file into smaller chunks and support isn't not responding to my request.

As a last ditch attempt I figure I can maybe set the memory to unlimited in PHP/MySQL but some how control the RAM the processes get and hopefully virtual memory will do the trick.

Cheers,
Alex

[Weirdan]

ulimit -m
cpulimit

[alex.barylski]

Thanks you for that.

So from quickly reading the docs, I should login as a regular user, set the CPU limit with the-m option to limit memory consumption? Do I need to call these or added these in a bash script or can I call them in a one off setting (I don't really need them after I run this spider program)?

Secondly, while the php.ini for CLI is set to unlimited memory use (script was crashing due to memory shortage which setting to -1 fixed) will limiting the system memory cause the same affect or will it just take longer?

thanks Weirdan

Cheers,
Alex

[Weirdan]

as for ulimit - you just set limits, then run your program:

Code: Select all

~$ ulimit -m $((1024*1024))
~$ nohup php path/to/script.php &
~$ ulimit -m unlimited

ulimit -m limits resident memory size for a program, so theoretically when the program exceeds this limit it should just start swapping.

as for cpulimit - I've never used it, so you're on your own here.