CLI script memory use

Whether you are using Linux on the desktop or as a server, it's still good that you're using Linux. Linux related questions go here.

Moderator: General Moderators

Post Reply
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

CLI script memory use

Post by alex.barylski »

Is it possible to run a CLI PHP script with unlimited memory from PHP perspective but restrict how much processor/memory it consume at the system level?

The reason I ask, is because I have a script which needs to spider a massive HTML file (over 130 MB). The spidering program doesn't seem to work when I partition the massive file into smaller chunks and support isn't not responding to my request.

As a last ditch attempt I figure I can maybe set the memory to unlimited in PHP/MySQL but some how control the RAM the processes get and hopefully virtual memory will do the trick.

Cheers,
Alex
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: CLI script memory use

Post by Weirdan »

ulimit -m
cpulimit
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Re: CLI script memory use

Post by alex.barylski »

Thanks you for that.

So from quickly reading the docs, I should login as a regular user, set the CPU limit with the-m option to limit memory consumption? Do I need to call these or added these in a bash script or can I call them in a one off setting (I don't really need them after I run this spider program)?

Secondly, while the php.ini for CLI is set to unlimited memory use (script was crashing due to memory shortage which setting to -1 fixed) will limiting the system memory cause the same affect or will it just take longer?

thanks Weirdan

Cheers,
Alex
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: CLI script memory use

Post by Weirdan »

as for ulimit - you just set limits, then run your program:

Code: Select all

~$ ulimit -m $((1024*1024))
~$ nohup php path/to/script.php &
~$ ulimit -m unlimited
ulimit -m limits resident memory size for a program, so theoretically when the program exceeds this limit it should just start swapping.

as for cpulimit - I've never used it, so you're on your own here.
Post Reply