Page 1 of 1

Can I use pcntl_fork() to speed up file reads/writes?

Posted: Thu Jul 29, 2010 4:14 pm
by j2010
I have a script that parses text files and inserts their contents into a local MySQL database. By using forking in PHP can I reduce the processing time by parsing multiple files simultaneously? Or am I misunderstanding the purpose of the function? Thanks!

Re: Can I use pcntl_fork() to speed up file reads/writes?

Posted: Thu Jul 29, 2010 7:48 pm
by Weirdan
Forking (or, more generally, running multiple instances simultaneously) helps to speed up processing when most of the time the script is waiting for some external resource, however it doesn't help if all script instances accessing the same resource and the resource's performance is already topped. Since it's usually quite easy to saturate hdd with reads using even single script instance it most likely won't help (unless different script instances are reading from different physical disks). Similarly, if they all are inserting data into the same table you're not likely to get much performance benefits from running multiple processing threads (and, most likely, you'd get decreased performance as a result of lock contention).

However don't take my words for granted and test it for yourself. Assuming you have a command line script that takes file name argument, you can easily run several script instances in parallel using this simple bash command line:

Code: Select all

for i in 1 2 3 4; do nohup php /path/to/script.php /path/to/file$i.txt 2>&1 > /path/to/logfile$i.log &; done;
it will spawn 4 script instances reading from file1.txt, file2.txt, etc.