Page 1 of 2

Coding for speed, and dodging timeouts

Posted: Thu May 04, 2006 2:50 pm
by cj5
The more and more projects I work on, the bigger they get, and the more demanding my code gets on the script engine. A recent project I am working on has taught me a serious lesson in learning how to tighten up my code some more so that it doesn't run like a pig through water. I am having a real tough time with this, and a lot of sites I googled, don't really address my speed issues. Where does one turn to find guidance in scripting efficient and non-memory intensive code? Any thoughts? Is PHP incapable of performing like lower level languages like C?

Posted: Thu May 04, 2006 3:04 pm
by Christopher
Optimization is the topic from Hades. There are a bunch of different kinds of optimizations and I really don't think you will get anything but religion unless you post some code so we can see what you are trying to do. That would include the version of PHP, database and the type and size of web server.

Posted: Thu May 04, 2006 3:25 pm
by cj5
Well, it's more a theoretical question. I don't have any code in mind, but I am tossing the topic out there to see what resources people have used to tighten up their code. Religion?! I don't understand what you're saying. Also, I use various versions of everything, depending on who I am working for, or who hosts their environment. I have written code for PHP 3,4, and 5. Used MySQL 4 and 5, and have developed on various platforms from like Mac, Windows, and *nix. It's not a specific question, it's more of an idea grabber. Get it?

Also, I just came across this online, sort of what I was looking for.

Read Me!

Posted: Thu May 04, 2006 3:33 pm
by Christopher
We theoretically it is just loading less data, less often, writing faster queries, and choosing good algorithms. I notice on your home page that you load a lot of CSS and Javascript on every page, but not much if it is used. A Response class can help solve that because each request only specifies what it needs to use.

By religion I mean strong programming opinions -- which are hard to find around here. ;)

PS - I notice you have a flash MP3 player on your site. I recently asked about that here. Any tips or code for me? I noticed that it takes a while to load the files. Slow server? Big files? Can they be streamed?

Posted: Thu May 04, 2006 3:44 pm
by Maugrim_The_Reaper
There are varying thoughts on optimisation in PHP - some look at small details in PHP function/language use, others at database optimisation, others talk about lots of caching, others think PHP should be optimised by not using PHP to start with, others...

End of the day, optimisation becomes a function of your experience. There are so many optimisation approaches there's no single silver bullet. You need to know your hardware, database, webserver and PHP code and how they all relate to one another. Faster PHP code by the way is not dependent solely on the code itself. Tweaking the database (for example in MySQL4/5 using InnoDB, and enabling a query cache of at least 16M) can increase performance without a single change in your PHP code. It's a big area you need to read up on... Another term to google is "scalable php applications" - usually gets you more non-PHP suggestions.

I think aborint is right about is being a religion to some - some do defend their optimisation approaches with a lot of zeal. Pity really.

Posted: Thu May 04, 2006 5:30 pm
by cj5
I understand your points of view, totally. When I approach PHP programming, I am very naive to the environment conditions that the PHP script engine is being utilized on. I suppose from what you guys mentioned, that PHP is too dependent on other factors within its environment to be a self-optimizing programming language. Very little of my coding is web-based recently. My projects usually involve delving into large file parsing, legacy system upgrading for web-based application, etc... This processing is hefty, and in most cases, leaves my clients wondering why it takes so long to run a program. I hope that I am not doing something wrong, and maybe that I am missing out on a key approach to strictly best PHP practices. This leads me to think that I should start looking to another language by which to accomplish my needs.

Posted: Thu May 04, 2006 7:01 pm
by AKA Panama Jack
It basically comes down to a learning experience for most people. Some PHP functions are faster than others depending upon how you use them. Also, just throwing a lot of code at a problem is usually the first mistake many programmers make. :)

As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.

To give you an example of how you can increase execution speed by changing how you program...

Code: Select all

for( $i = 0; $i < count($values); $i++)
{
// do stuff
}
The above is a nice fast programming shortcut but slower than snot when it comes to execution.

Code: Select all

$count_total = count($values);

for( $i = 0; $i < $count_total; $i++)
{
// do stuff
}
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration.

This is one of those cases where more code is faster than less code. :)

The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.

Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything. ;)

Re: Coding for speed, and dodging timeouts

Posted: Thu May 04, 2006 8:21 pm
by alex.barylski
cj5 wrote:The more and more projects I work on, the bigger they get, and the more demanding my code gets on the script engine. A recent project I am working on has taught me a serious lesson in learning how to tighten up my code some more so that it doesn't run like a pig through water. I am having a real tough time with this, and a lot of sites I googled, don't really address my speed issues. Where does one turn to find guidance in scripting efficient and non-memory intensive code? Any thoughts? Is PHP incapable of performing like lower level languages like C?
Yes...practically impossible as the nature of an interpreted language doesn't allow it.

Flexibility and ease of use come at a cost...that cost in this case is performance ;)

Even compiled into byte code...it's not practical...as byte code is still machine agostic...

I should note that PHP as a language is capable of executing at break neck speed, but what slows PHP down is the context is runs under...which is typically that of interpretation...in complied machine specific code...PHP could run at C's speed no problem :)

It's just not practical as PHP typically isn't compiled in the true sense of the word...

Cheers :)

Posted: Thu May 04, 2006 8:34 pm
by alex.barylski
AKA Panama Jack wrote:It basically comes down to a learning experience for most people. Some PHP functions are faster than others depending upon how you use them. Also, just throwing a lot of code at a problem is usually the first mistake many programmers make. :)

As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.

To give you an example of how you can increase execution speed by changing how you program...

Code: Select all

for( $i = 0; $i < count($values); $i++)
{
// do stuff
}
The above is a nice fast programming shortcut but slower than snot when it comes to execution.

Code: Select all

$count_total = count($values);

for( $i = 0; $i < $count_total; $i++)
{
// do stuff
}
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration.

This is one of those cases where more code is faster than less code. :)

The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.

Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything. ;)
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration
I read about that a few years back...I figured they would have solved that by now??? :?

Still I always try and remember to keep count() out of my loops :)

Posted: Thu May 04, 2006 8:40 pm
by Christopher
cj5 wrote:My projects usually involve delving into large file parsing, legacy system upgrading for web-based application, etc... This processing is hefty, and in most cases, leaves my clients wondering why it takes so long to run a program. I hope that I am not doing something wrong, and maybe that I am missing out on a key approach to strictly best PHP practices. This leads me to think that I should start looking to another language by which to accomplish my needs.
PHP is not really a systems language. While slower, I have not found it order of magnitude slower than other languages. My experience is that unless you have picked a really bad algorithm then it is usually some sub-system or the connection to that sub-system that is the cause of the slowness. Can you give us some examples?

Posted: Thu May 04, 2006 8:45 pm
by s.dot
I have a pretty server intense set of scripts, and I'm currently redoing them and I'm doing lots of things to speed up scripts.

Using cronjobs on things that won't change on every single page view
Storing data into arrays and referencing that array instead of sticking queries inside loops
If you know how many records a specific query is going to return, use the LIMIT clause (why scan the whole table?)
Unset variables, arrays, and objects that are no longer going to be used
Free mysql result sets
Reduce calls to redundant functions ( small example, why call echo 100 times when you can do it just once? )

I'm sure there are more, but these are ones that I have started doing and am noticing page loads that are in the thousandths of a second in loading time... with a fairly decent sized database.

Posted: Thu May 04, 2006 8:45 pm
by AKA Panama Jack
Hockey wrote:I read about that a few years back...I figured they would have solved that by now??? :?

Still I always try and remember to keep count() out of my loops :)
Yeah, never been fixed even in the latst versions of PHP 5. It's amazing how much that will slow down execution. Especially if you have a large array with thousands of elements to process.

Posted: Thu May 04, 2006 8:51 pm
by alex.barylski
AKA Panama Jack wrote:
Hockey wrote:I read about that a few years back...I figured they would have solved that by now??? :?

Still I always try and remember to keep count() out of my loops :)
Yeah, never been fixed even in the latst versions of PHP 5. It's amazing how much that will slow down execution. Especially if you have a large array with thousands of elements to process.
When I read I just assumed the author was correct, never did bother testing it though...as his explanation made enough sense to convince me :)

An optimization which I discovered...that also deals with arrays...

I personally found that setting an array element to FALSE instead of unset() and then calling:

Code: Select all

array_values(array_filter($arr))
The above cleans recalculates the indicies and removes all elements which are FALSE...in the reverse order, but still :)

Cheers :)

Posted: Fri May 05, 2006 8:49 am
by cj5
Now these are the answers I was seeking. From a code point of view, these are great points. My issue arose with two scripts I was working on. One script I am currently working on is a shapefile parser. Now, if you are not familiar with shapefiles, they are typically very bulky, containing a huge amount of data (i.e. a map of the US and all it's states include all of the coordinate mappings for each state, in order to plot it's shape). In most cases the script will run long but not timeout, even with streamlined code in place. My worry is when I use GD to draw the shapes based on the mappings, that the script will become even more load intensive, and will become less practical.
I have been able to speed up the process some by heeding some advice in this forum topic, and by turning output_buffering on in the ini file. I hear there are also some Apache conf tweaks you can use too.

Posted: Fri May 05, 2006 12:09 pm
by alex.barylski
Drawing maps like that of the USA would be an intensive process :P

So I can see where your concern comes in...

I have plenty of experience rendering maps...so I ask you this?

Perhaps obvious but a huge performance increase...

When you render a map do you consider scale? Are you implementing code to clip any drawing operations outside of the current viewport?

If you have zoomed into say Texas, there is no point to rendering North Dakota...

Drawing operations of anykind or typically much more resource intensive than others...so having code which says...wait no reason to render this coordinate point on a map because it's outside of the viewport...makes a big difference :)

Also, you can try logically ordering your coordinate points into sub-sections...

ie: Keep your coordniates for one state in one array and another state in another array and so on...

This way you can also imlement code which avoids iterating massive amounts of coordinates which are never even rendered...

Just some points t consider if you haven't done so already :)

Cheers :)