Coding for speed, and dodging timeouts
Moderator: General Moderators
Coding for speed, and dodging timeouts
The more and more projects I work on, the bigger they get, and the more demanding my code gets on the script engine. A recent project I am working on has taught me a serious lesson in learning how to tighten up my code some more so that it doesn't run like a pig through water. I am having a real tough time with this, and a lot of sites I googled, don't really address my speed issues. Where does one turn to find guidance in scripting efficient and non-memory intensive code? Any thoughts? Is PHP incapable of performing like lower level languages like C?
- Christopher
- Site Administrator
- Posts: 13596
- Joined: Wed Aug 25, 2004 7:54 pm
- Location: New York, NY, US
Optimization is the topic from Hades. There are a bunch of different kinds of optimizations and I really don't think you will get anything but religion unless you post some code so we can see what you are trying to do. That would include the version of PHP, database and the type and size of web server.
(#10850)
Well, it's more a theoretical question. I don't have any code in mind, but I am tossing the topic out there to see what resources people have used to tighten up their code. Religion?! I don't understand what you're saying. Also, I use various versions of everything, depending on who I am working for, or who hosts their environment. I have written code for PHP 3,4, and 5. Used MySQL 4 and 5, and have developed on various platforms from like Mac, Windows, and *nix. It's not a specific question, it's more of an idea grabber. Get it?
Also, I just came across this online, sort of what I was looking for.
Read Me!
Also, I just came across this online, sort of what I was looking for.
Read Me!
- Christopher
- Site Administrator
- Posts: 13596
- Joined: Wed Aug 25, 2004 7:54 pm
- Location: New York, NY, US
We theoretically it is just loading less data, less often, writing faster queries, and choosing good algorithms. I notice on your home page that you load a lot of CSS and Javascript on every page, but not much if it is used. A Response class can help solve that because each request only specifies what it needs to use.
By religion I mean strong programming opinions -- which are hard to find around here.
PS - I notice you have a flash MP3 player on your site. I recently asked about that here. Any tips or code for me? I noticed that it takes a while to load the files. Slow server? Big files? Can they be streamed?
By religion I mean strong programming opinions -- which are hard to find around here.
PS - I notice you have a flash MP3 player on your site. I recently asked about that here. Any tips or code for me? I noticed that it takes a while to load the files. Slow server? Big files? Can they be streamed?
(#10850)
- Maugrim_The_Reaper
- DevNet Master
- Posts: 2704
- Joined: Tue Nov 02, 2004 5:43 am
- Location: Ireland
There are varying thoughts on optimisation in PHP - some look at small details in PHP function/language use, others at database optimisation, others talk about lots of caching, others think PHP should be optimised by not using PHP to start with, others...
End of the day, optimisation becomes a function of your experience. There are so many optimisation approaches there's no single silver bullet. You need to know your hardware, database, webserver and PHP code and how they all relate to one another. Faster PHP code by the way is not dependent solely on the code itself. Tweaking the database (for example in MySQL4/5 using InnoDB, and enabling a query cache of at least 16M) can increase performance without a single change in your PHP code. It's a big area you need to read up on... Another term to google is "scalable php applications" - usually gets you more non-PHP suggestions.
I think aborint is right about is being a religion to some - some do defend their optimisation approaches with a lot of zeal. Pity really.
End of the day, optimisation becomes a function of your experience. There are so many optimisation approaches there's no single silver bullet. You need to know your hardware, database, webserver and PHP code and how they all relate to one another. Faster PHP code by the way is not dependent solely on the code itself. Tweaking the database (for example in MySQL4/5 using InnoDB, and enabling a query cache of at least 16M) can increase performance without a single change in your PHP code. It's a big area you need to read up on... Another term to google is "scalable php applications" - usually gets you more non-PHP suggestions.
I think aborint is right about is being a religion to some - some do defend their optimisation approaches with a lot of zeal. Pity really.
I understand your points of view, totally. When I approach PHP programming, I am very naive to the environment conditions that the PHP script engine is being utilized on. I suppose from what you guys mentioned, that PHP is too dependent on other factors within its environment to be a self-optimizing programming language. Very little of my coding is web-based recently. My projects usually involve delving into large file parsing, legacy system upgrading for web-based application, etc... This processing is hefty, and in most cases, leaves my clients wondering why it takes so long to run a program. I hope that I am not doing something wrong, and maybe that I am missing out on a key approach to strictly best PHP practices. This leads me to think that I should start looking to another language by which to accomplish my needs.
- AKA Panama Jack
- Forum Regular
- Posts: 878
- Joined: Mon Nov 14, 2005 4:21 pm
It basically comes down to a learning experience for most people. Some PHP functions are faster than others depending upon how you use them. Also, just throwing a lot of code at a problem is usually the first mistake many programmers make. 
As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.
To give you an example of how you can increase execution speed by changing how you program...
The above is a nice fast programming shortcut but slower than snot when it comes to execution.
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration.
This is one of those cases where more code is faster than less code.
The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.
Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything.
As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.
To give you an example of how you can increase execution speed by changing how you program...
Code: Select all
for( $i = 0; $i < count($values); $i++)
{
// do stuff
}Code: Select all
$count_total = count($values);
for( $i = 0; $i < $count_total; $i++)
{
// do stuff
}This is one of those cases where more code is faster than less code.
The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.
Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything.
-
alex.barylski
- DevNet Evangelist
- Posts: 6267
- Joined: Tue Dec 21, 2004 5:00 pm
- Location: Winnipeg
Re: Coding for speed, and dodging timeouts
Yes...practically impossible as the nature of an interpreted language doesn't allow it.cj5 wrote:The more and more projects I work on, the bigger they get, and the more demanding my code gets on the script engine. A recent project I am working on has taught me a serious lesson in learning how to tighten up my code some more so that it doesn't run like a pig through water. I am having a real tough time with this, and a lot of sites I googled, don't really address my speed issues. Where does one turn to find guidance in scripting efficient and non-memory intensive code? Any thoughts? Is PHP incapable of performing like lower level languages like C?
Flexibility and ease of use come at a cost...that cost in this case is performance
Even compiled into byte code...it's not practical...as byte code is still machine agostic...
I should note that PHP as a language is capable of executing at break neck speed, but what slows PHP down is the context is runs under...which is typically that of interpretation...in complied machine specific code...PHP could run at C's speed no problem
It's just not practical as PHP typically isn't compiled in the true sense of the word...
Cheers
-
alex.barylski
- DevNet Evangelist
- Posts: 6267
- Joined: Tue Dec 21, 2004 5:00 pm
- Location: Winnipeg
AKA Panama Jack wrote:It basically comes down to a learning experience for most people. Some PHP functions are faster than others depending upon how you use them. Also, just throwing a lot of code at a problem is usually the first mistake many programmers make.
As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.
To give you an example of how you can increase execution speed by changing how you program...
The above is a nice fast programming shortcut but slower than snot when it comes to execution.Code: Select all
for( $i = 0; $i < count($values); $i++) { // do stuff }
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration.Code: Select all
$count_total = count($values); for( $i = 0; $i < $count_total; $i++) { // do stuff }
This is one of those cases where more code is faster than less code.
The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.
Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything.
I read about that a few years back...I figured they would have solved that by now???The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration
Still I always try and remember to keep count() out of my loops
- Christopher
- Site Administrator
- Posts: 13596
- Joined: Wed Aug 25, 2004 7:54 pm
- Location: New York, NY, US
PHP is not really a systems language. While slower, I have not found it order of magnitude slower than other languages. My experience is that unless you have picked a really bad algorithm then it is usually some sub-system or the connection to that sub-system that is the cause of the slowness. Can you give us some examples?cj5 wrote:My projects usually involve delving into large file parsing, legacy system upgrading for web-based application, etc... This processing is hefty, and in most cases, leaves my clients wondering why it takes so long to run a program. I hope that I am not doing something wrong, and maybe that I am missing out on a key approach to strictly best PHP practices. This leads me to think that I should start looking to another language by which to accomplish my needs.
(#10850)
I have a pretty server intense set of scripts, and I'm currently redoing them and I'm doing lots of things to speed up scripts.
Using cronjobs on things that won't change on every single page view
Storing data into arrays and referencing that array instead of sticking queries inside loops
If you know how many records a specific query is going to return, use the LIMIT clause (why scan the whole table?)
Unset variables, arrays, and objects that are no longer going to be used
Free mysql result sets
Reduce calls to redundant functions ( small example, why call echo 100 times when you can do it just once? )
I'm sure there are more, but these are ones that I have started doing and am noticing page loads that are in the thousandths of a second in loading time... with a fairly decent sized database.
Using cronjobs on things that won't change on every single page view
Storing data into arrays and referencing that array instead of sticking queries inside loops
If you know how many records a specific query is going to return, use the LIMIT clause (why scan the whole table?)
Unset variables, arrays, and objects that are no longer going to be used
Free mysql result sets
Reduce calls to redundant functions ( small example, why call echo 100 times when you can do it just once? )
I'm sure there are more, but these are ones that I have started doing and am noticing page loads that are in the thousandths of a second in loading time... with a fairly decent sized database.
Set Search Time - A google chrome extension. When you search only results from the past year (or set time period) are displayed. Helps tremendously when using new technologies to avoid outdated results.
- AKA Panama Jack
- Forum Regular
- Posts: 878
- Joined: Mon Nov 14, 2005 4:21 pm
Yeah, never been fixed even in the latst versions of PHP 5. It's amazing how much that will slow down execution. Especially if you have a large array with thousands of elements to process.Hockey wrote:I read about that a few years back...I figured they would have solved that by now???![]()
Still I always try and remember to keep count() out of my loops
-
alex.barylski
- DevNet Evangelist
- Posts: 6267
- Joined: Tue Dec 21, 2004 5:00 pm
- Location: Winnipeg
When I read I just assumed the author was correct, never did bother testing it though...as his explanation made enough sense to convince meAKA Panama Jack wrote:Yeah, never been fixed even in the latst versions of PHP 5. It's amazing how much that will slow down execution. Especially if you have a large array with thousands of elements to process.Hockey wrote:I read about that a few years back...I figured they would have solved that by now???![]()
Still I always try and remember to keep count() out of my loops
An optimization which I discovered...that also deals with arrays...
I personally found that setting an array element to FALSE instead of unset() and then calling:
Code: Select all
array_values(array_filter($arr))Cheers
Now these are the answers I was seeking. From a code point of view, these are great points. My issue arose with two scripts I was working on. One script I am currently working on is a shapefile parser. Now, if you are not familiar with shapefiles, they are typically very bulky, containing a huge amount of data (i.e. a map of the US and all it's states include all of the coordinate mappings for each state, in order to plot it's shape). In most cases the script will run long but not timeout, even with streamlined code in place. My worry is when I use GD to draw the shapes based on the mappings, that the script will become even more load intensive, and will become less practical.
I have been able to speed up the process some by heeding some advice in this forum topic, and by turning output_buffering on in the ini file. I hear there are also some Apache conf tweaks you can use too.
I have been able to speed up the process some by heeding some advice in this forum topic, and by turning output_buffering on in the ini file. I hear there are also some Apache conf tweaks you can use too.
-
alex.barylski
- DevNet Evangelist
- Posts: 6267
- Joined: Tue Dec 21, 2004 5:00 pm
- Location: Winnipeg
Drawing maps like that of the USA would be an intensive process 
So I can see where your concern comes in...
I have plenty of experience rendering maps...so I ask you this?
Perhaps obvious but a huge performance increase...
When you render a map do you consider scale? Are you implementing code to clip any drawing operations outside of the current viewport?
If you have zoomed into say Texas, there is no point to rendering North Dakota...
Drawing operations of anykind or typically much more resource intensive than others...so having code which says...wait no reason to render this coordinate point on a map because it's outside of the viewport...makes a big difference
Also, you can try logically ordering your coordinate points into sub-sections...
ie: Keep your coordniates for one state in one array and another state in another array and so on...
This way you can also imlement code which avoids iterating massive amounts of coordinates which are never even rendered...
Just some points t consider if you haven't done so already
Cheers
So I can see where your concern comes in...
I have plenty of experience rendering maps...so I ask you this?
Perhaps obvious but a huge performance increase...
When you render a map do you consider scale? Are you implementing code to clip any drawing operations outside of the current viewport?
If you have zoomed into say Texas, there is no point to rendering North Dakota...
Drawing operations of anykind or typically much more resource intensive than others...so having code which says...wait no reason to render this coordinate point on a map because it's outside of the viewport...makes a big difference
Also, you can try logically ordering your coordinate points into sub-sections...
ie: Keep your coordniates for one state in one array and another state in another array and so on...
This way you can also imlement code which avoids iterating massive amounts of coordinates which are never even rendered...
Just some points t consider if you haven't done so already
Cheers