Coding for speed, and dodging timeouts

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

User avatar
cj5
Forum Commoner
Posts: 60
Joined: Tue Jan 17, 2006 3:38 pm
Location: Long Island, NY, USA

Coding for speed, and dodging timeouts

Post by cj5 »

The more and more projects I work on, the bigger they get, and the more demanding my code gets on the script engine. A recent project I am working on has taught me a serious lesson in learning how to tighten up my code some more so that it doesn't run like a pig through water. I am having a real tough time with this, and a lot of sites I googled, don't really address my speed issues. Where does one turn to find guidance in scripting efficient and non-memory intensive code? Any thoughts? Is PHP incapable of performing like lower level languages like C?
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

Optimization is the topic from Hades. There are a bunch of different kinds of optimizations and I really don't think you will get anything but religion unless you post some code so we can see what you are trying to do. That would include the version of PHP, database and the type and size of web server.
(#10850)
User avatar
cj5
Forum Commoner
Posts: 60
Joined: Tue Jan 17, 2006 3:38 pm
Location: Long Island, NY, USA

Post by cj5 »

Well, it's more a theoretical question. I don't have any code in mind, but I am tossing the topic out there to see what resources people have used to tighten up their code. Religion?! I don't understand what you're saying. Also, I use various versions of everything, depending on who I am working for, or who hosts their environment. I have written code for PHP 3,4, and 5. Used MySQL 4 and 5, and have developed on various platforms from like Mac, Windows, and *nix. It's not a specific question, it's more of an idea grabber. Get it?

Also, I just came across this online, sort of what I was looking for.

Read Me!
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

We theoretically it is just loading less data, less often, writing faster queries, and choosing good algorithms. I notice on your home page that you load a lot of CSS and Javascript on every page, but not much if it is used. A Response class can help solve that because each request only specifies what it needs to use.

By religion I mean strong programming opinions -- which are hard to find around here. ;)

PS - I notice you have a flash MP3 player on your site. I recently asked about that here. Any tips or code for me? I noticed that it takes a while to load the files. Slow server? Big files? Can they be streamed?
(#10850)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

There are varying thoughts on optimisation in PHP - some look at small details in PHP function/language use, others at database optimisation, others talk about lots of caching, others think PHP should be optimised by not using PHP to start with, others...

End of the day, optimisation becomes a function of your experience. There are so many optimisation approaches there's no single silver bullet. You need to know your hardware, database, webserver and PHP code and how they all relate to one another. Faster PHP code by the way is not dependent solely on the code itself. Tweaking the database (for example in MySQL4/5 using InnoDB, and enabling a query cache of at least 16M) can increase performance without a single change in your PHP code. It's a big area you need to read up on... Another term to google is "scalable php applications" - usually gets you more non-PHP suggestions.

I think aborint is right about is being a religion to some - some do defend their optimisation approaches with a lot of zeal. Pity really.
User avatar
cj5
Forum Commoner
Posts: 60
Joined: Tue Jan 17, 2006 3:38 pm
Location: Long Island, NY, USA

Post by cj5 »

I understand your points of view, totally. When I approach PHP programming, I am very naive to the environment conditions that the PHP script engine is being utilized on. I suppose from what you guys mentioned, that PHP is too dependent on other factors within its environment to be a self-optimizing programming language. Very little of my coding is web-based recently. My projects usually involve delving into large file parsing, legacy system upgrading for web-based application, etc... This processing is hefty, and in most cases, leaves my clients wondering why it takes so long to run a program. I hope that I am not doing something wrong, and maybe that I am missing out on a key approach to strictly best PHP practices. This leads me to think that I should start looking to another language by which to accomplish my needs.
User avatar
AKA Panama Jack
Forum Regular
Posts: 878
Joined: Mon Nov 14, 2005 4:21 pm

Post by AKA Panama Jack »

It basically comes down to a learning experience for most people. Some PHP functions are faster than others depending upon how you use them. Also, just throwing a lot of code at a problem is usually the first mistake many programmers make. :)

As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.

To give you an example of how you can increase execution speed by changing how you program...

Code: Select all

for( $i = 0; $i < count($values); $i++)
{
// do stuff
}
The above is a nice fast programming shortcut but slower than snot when it comes to execution.

Code: Select all

$count_total = count($values);

for( $i = 0; $i < $count_total; $i++)
{
// do stuff
}
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration.

This is one of those cases where more code is faster than less code. :)

The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.

Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything. ;)
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Re: Coding for speed, and dodging timeouts

Post by alex.barylski »

cj5 wrote:The more and more projects I work on, the bigger they get, and the more demanding my code gets on the script engine. A recent project I am working on has taught me a serious lesson in learning how to tighten up my code some more so that it doesn't run like a pig through water. I am having a real tough time with this, and a lot of sites I googled, don't really address my speed issues. Where does one turn to find guidance in scripting efficient and non-memory intensive code? Any thoughts? Is PHP incapable of performing like lower level languages like C?
Yes...practically impossible as the nature of an interpreted language doesn't allow it.

Flexibility and ease of use come at a cost...that cost in this case is performance ;)

Even compiled into byte code...it's not practical...as byte code is still machine agostic...

I should note that PHP as a language is capable of executing at break neck speed, but what slows PHP down is the context is runs under...which is typically that of interpretation...in complied machine specific code...PHP could run at C's speed no problem :)

It's just not practical as PHP typically isn't compiled in the true sense of the word...

Cheers :)
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

AKA Panama Jack wrote:It basically comes down to a learning experience for most people. Some PHP functions are faster than others depending upon how you use them. Also, just throwing a lot of code at a problem is usually the first mistake many programmers make. :)

As a general rule the less code it takes to accomplish your task the lower the server load and faster it will execute. That's not always the case as I have seen some 2-3 line code that were server killers. But as a General Rule its true.

To give you an example of how you can increase execution speed by changing how you program...

Code: Select all

for( $i = 0; $i < count($values); $i++)
{
// do stuff
}
The above is a nice fast programming shortcut but slower than snot when it comes to execution.

Code: Select all

$count_total = count($values);

for( $i = 0; $i < $count_total; $i++)
{
// do stuff
}
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration.

This is one of those cases where more code is faster than less code. :)

The other thing is to make your php code as MODULAR as possible. Having one massive PHP program with a bunch of branches, switches and classes is going to be very slow and resource intensive. Having everything broken down into modules and loading only what is needed for that particular web page will reduce execution time and server load greatly.

Unfortunately there isn't one place where you can go to find out what PHP functions are faster than others for similar operations or code combinations like I listed above. Most of it is gained through personal experience and trial and error. Plus, some of us benchmark everything. ;)
The above will execute many, many, MANY times faster than the first example. In the first example each iteration of the for loop will cause the count to recalculate the number of elements in the $values array. By moving the count outside the for loop it is only executed once instead of once for every iteration
I read about that a few years back...I figured they would have solved that by now??? :?

Still I always try and remember to keep count() out of my loops :)
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

cj5 wrote:My projects usually involve delving into large file parsing, legacy system upgrading for web-based application, etc... This processing is hefty, and in most cases, leaves my clients wondering why it takes so long to run a program. I hope that I am not doing something wrong, and maybe that I am missing out on a key approach to strictly best PHP practices. This leads me to think that I should start looking to another language by which to accomplish my needs.
PHP is not really a systems language. While slower, I have not found it order of magnitude slower than other languages. My experience is that unless you have picked a really bad algorithm then it is usually some sub-system or the connection to that sub-system that is the cause of the slowness. Can you give us some examples?
(#10850)
User avatar
s.dot
Tranquility In Moderation
Posts: 5001
Joined: Sun Feb 06, 2005 7:18 pm
Location: Indiana

Post by s.dot »

I have a pretty server intense set of scripts, and I'm currently redoing them and I'm doing lots of things to speed up scripts.

Using cronjobs on things that won't change on every single page view
Storing data into arrays and referencing that array instead of sticking queries inside loops
If you know how many records a specific query is going to return, use the LIMIT clause (why scan the whole table?)
Unset variables, arrays, and objects that are no longer going to be used
Free mysql result sets
Reduce calls to redundant functions ( small example, why call echo 100 times when you can do it just once? )

I'm sure there are more, but these are ones that I have started doing and am noticing page loads that are in the thousandths of a second in loading time... with a fairly decent sized database.
Set Search Time - A google chrome extension. When you search only results from the past year (or set time period) are displayed. Helps tremendously when using new technologies to avoid outdated results.
User avatar
AKA Panama Jack
Forum Regular
Posts: 878
Joined: Mon Nov 14, 2005 4:21 pm

Post by AKA Panama Jack »

Hockey wrote:I read about that a few years back...I figured they would have solved that by now??? :?

Still I always try and remember to keep count() out of my loops :)
Yeah, never been fixed even in the latst versions of PHP 5. It's amazing how much that will slow down execution. Especially if you have a large array with thousands of elements to process.
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

AKA Panama Jack wrote:
Hockey wrote:I read about that a few years back...I figured they would have solved that by now??? :?

Still I always try and remember to keep count() out of my loops :)
Yeah, never been fixed even in the latst versions of PHP 5. It's amazing how much that will slow down execution. Especially if you have a large array with thousands of elements to process.
When I read I just assumed the author was correct, never did bother testing it though...as his explanation made enough sense to convince me :)

An optimization which I discovered...that also deals with arrays...

I personally found that setting an array element to FALSE instead of unset() and then calling:

Code: Select all

array_values(array_filter($arr))
The above cleans recalculates the indicies and removes all elements which are FALSE...in the reverse order, but still :)

Cheers :)
User avatar
cj5
Forum Commoner
Posts: 60
Joined: Tue Jan 17, 2006 3:38 pm
Location: Long Island, NY, USA

Post by cj5 »

Now these are the answers I was seeking. From a code point of view, these are great points. My issue arose with two scripts I was working on. One script I am currently working on is a shapefile parser. Now, if you are not familiar with shapefiles, they are typically very bulky, containing a huge amount of data (i.e. a map of the US and all it's states include all of the coordinate mappings for each state, in order to plot it's shape). In most cases the script will run long but not timeout, even with streamlined code in place. My worry is when I use GD to draw the shapes based on the mappings, that the script will become even more load intensive, and will become less practical.
I have been able to speed up the process some by heeding some advice in this forum topic, and by turning output_buffering on in the ini file. I hear there are also some Apache conf tweaks you can use too.
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

Drawing maps like that of the USA would be an intensive process :P

So I can see where your concern comes in...

I have plenty of experience rendering maps...so I ask you this?

Perhaps obvious but a huge performance increase...

When you render a map do you consider scale? Are you implementing code to clip any drawing operations outside of the current viewport?

If you have zoomed into say Texas, there is no point to rendering North Dakota...

Drawing operations of anykind or typically much more resource intensive than others...so having code which says...wait no reason to render this coordinate point on a map because it's outside of the viewport...makes a big difference :)

Also, you can try logically ordering your coordinate points into sub-sections...

ie: Keep your coordniates for one state in one array and another state in another array and so on...

This way you can also imlement code which avoids iterating massive amounts of coordinates which are never even rendered...

Just some points t consider if you haven't done so already :)

Cheers :)
Post Reply