Page 3 of 4

Posted: Fri Mar 26, 2004 1:30 pm
by aleigh
penguinboy wrote:
All the Bosses I've worked for have had one thing in common;

a motto:

'Get it done quick.
Get it done now.
Optimize later.'

And they only(read: never) support optimizing when performance becomes a problem.
I wonder if that's good or bad. I work in an environment where we do hundreds of millions of transactions per day so it's really a bottom line issue. Sort of like if you could take 1 cent from everyone in the world, saving seemingly trivial amounts of speed by just writing better programs adds up.

Then there's the little tin god problem though, haha. Can't take it too far, like anything.

Posted: Fri Mar 26, 2004 1:35 pm
by Buddha443556
McGruff wrote:We're talking specifically about php and web apps. Scripts will commonly take a couple of tenths of a second to execute. That itself is insignificant to the end user quite apart from the 8% speed gain (8% of that 0.2s) obtained from a script optimisation mentioned above. If your server is on its knees get new hardware.
It isn't insignificant if your receiving 200 million hits a day and neither are the cost of hardware or bandwidth at that scale. At that scale, every 0.1s is significant. Please show me a customer that doesn't want there site to be the next Google. Optimization and scalability go hand in hand.
McGruff wrote:There is not much to be had from optimising php.
That's because you see optimization as only part of the coding process.
McGruff wrote:Given that, and the mutability of scripts during the development process, it's not worth spending time on. Test when it's done and find out if you actually have a problem before trying to fix it.
I agree your not going to see your biggest gains from optimization in the development process, your biggest gain from optimization will come in the design process. But, I'm beginning to think you put no time into the design process. Correct me if I'm wrong. I don't care what mythodology your using that will lead to problems.
McGruff wrote:PS: I don't think my little story was insulting to anyone - it certainly wasn't meant to be.
Don't worry I'm thick thinned.

Posted: Fri Mar 26, 2004 1:39 pm
by McGruff
Buddha443556 wrote:It isn't insignificant if your receiving 200 million hits a day and neither are the cost of hardware or bandwidth at that scale.
Agreed - do the sums, work out the best approach.

Posted: Fri Mar 26, 2004 2:45 pm
by Roja
McGruff wrote:
Roja wrote: I disagree with that statement completely - the LESS well designed the app, the MORE benefit you can get from optimizations. If its well designed, its going to be VERY challenging to improve that.
I don't think you really meant to say that.. :)
On the contrary - I did.

Lets say a totally crap design is 10%. Then lets say you improve that 40% - now its 50%, which we will call "passably average".

To take an application from 50% (passably average) to 90% is completely and totally much harder than getting from crap to passable. Its a logarithmic progression at best - maybe even higher. The higher up the 'perfection' slope you get, the harder it is. At 90%, good luck for even a 1% improvement.

Thats why the context of the original poster matters.. if he's at 10%, I can make a great argument that my improvements would be faster, easier, and much more beneficial in terms of return on programmer time investment. If he's at 90%, I can absolutely guarantee that there is almost NOTHING my techniques would do to improve things, while moving towards a refactor/redesign would be extremely beneficial.

Thats the point I've been making since my first post - both have value, we just disagree on where they have value. You think OOP is better in more places, I think procedural fixes are better in more places. Simple diversity of opinion, and I do respect yours - although I have a different one. :)
McGruff wrote:
Roja wrote: If you don't have control over the hardware, and your problem is that the script is not fast enough - it is a direct solution to the problem at hand.
But you always have control of the hardware.
Not at all true - Did you read my post??!? I specifically explained why you don't. If you write your app for OTHER people to deploy - for example a backend code (database abstraction, login system, template system, etc), or even a game (other people run it on their servers), then you DO NOT have control over the hardware.

At best, the people DEPLOYING the game or the backend MIGHT - but the vast majority of the people deploying the two projects I mentioned DONT. They are admins running the games/backends on webhosting accounts, which they don't control the hardware for. Thus, in that situation, as the coder, I do not have control of the hardware, and neither do the people deploying the code. Hence, speed MATTERS.
McGruff wrote: Even if you can't physically get your hands on the box, you can upgrade your host plan or get a better host.
No, the admins deploying it can - and that means reducing the target audience. For example, if my game wont run on a 256mb hosting account, thats (off-the-wall guess) 20% less admins that can run my game. If it wont run on an account with MySQL (like if I switched to only supporting postgresql) suddenly, its 40% less admins that can run my game.

And your argument is that that is an improvement over spending time optimizing the code?
McGruff wrote: You have to keep a perspective: hardware is cheap, programmer's time is not.
And I say you need to consider the problem - in my projects, the hardware is a fixed commodity, where you have no control. The market dictates the common platform for webhosting - the larger the requirements, the less people will install it. Suddenly, the programmers time is much LESS valuable than the market share is. :)
McGruff wrote: I think we are starting to go round in circles a bit. I guess we'll have to leave others to judge for themselves.
I think in many ways, I agree totally.

We simply see things differently - you are more yin, I am more yang. No problems with that.

In the future, please focus on the issues at hand - the thread was about optimization. You have made valid points about how OOP can be used in that context, and I didnt disagree that it had a value. You however, did disagree that MY suggestions had any value.

My problem was that you took my ideas, implied that they were spaghetti coding, said that "its not the way its done", and then said "its not the way experienced programmers do it".

All of which does little to offer the original poster advice on how to improve his performance - but does reduce the value of what I offered the original poster. If a moderator will offer little in the way of constructive advice beyond "make it OOP", at least don't insult those of us offering concrete examples of how to improve things.

Posted: Fri Mar 26, 2004 3:34 pm
by McGruff
I think you took the Mr Spaghetti story the wrong way. If you read it again you'll see I was talking about a common attitude encountered in new programmers. I did not allude to any individual.

This forum gets a lot of visits from people fairly new to php. If I could achieve just one thing here it would be to help them make the step up from tangled-up code with not a function in sight (never mind a class), all in the global scope. We've all done it. Obsessive optimisation impedes that progress. Hence the story. I thought it might be midly humourous but there you go.

Sure you can find special cases where general statements fall down. There are less adversarial ways to bring these out. It's far too easy to bounce off each other on a discussion forum.

Of course if you write bad code it will need attention but usually what people are fretting over is not important.

I simply don't believe that swapping print for echo can give you an 8% improvement - or that an 8% improvement has much significance. It's important that members don't receive the wrong impression about the comparative behaviour of these fns.

Posted: Fri Mar 26, 2004 4:16 pm
by Buddha443556
McGruff wrote:This forum gets a lot of visits from people fairly new to php. If I could achieve just one thing here it would be to help them make the step up from tangled-up code with not a function in sight (never mind a class), all in the global scope.
We may not have much in common when it comes to programming, but I guess we're both here for the same reason to help others. :)

Posted: Fri Mar 26, 2004 4:27 pm
by markl999
If you care that much about speed, don't use PHP.

Ooops..did i just say that out loud :o
This forum gets a lot of visits from people fairly new to php.
I always find the hardest part of large forums like this with wide ranging user abilities is to keep exaplanations/examples simple and non technical without having the more knowledgeable people jump all over you for your technical inaccuracies. I feel the same as McGruff, there's no better feeling (for me anyway) than the penny dropping and suddenly something becomes clear, if you can help someone get there then it's a job well done imho.
</slightly off topic rant> ;)

Posted: Fri Mar 26, 2004 10:55 pm
by gaddimanish
Roja wrote:

I wasn't being specific actually - I was giving general recommendations. The original poster would really need to show the code (or at least give some more specific generalizations about what its doing) to give specific recommendations on how to improve it. I mentioned echo v. print. I also mentioned poor loops, sql queries, and less-optimal choices of functions.

Reply:

Main problem is that I am including one common file for all functions of my project in all pages.
It already crosses 3000+ lines of code.
I am also including file in which i have declared various variables and assign them values and using them in my pages .In this file I have declared 500 varaibles..I am including the same file in all the pages although i need only 5-10 variables in my page.

I am also using classes.SO i have cerated object for the same in my page.
I just want to know whether the slow speed of my page is sloa because of above reasons.

Generally I want to ask whether lengthy code and use of classes slows
down the speed or not

Manish

Posted: Fri Mar 26, 2004 11:42 pm
by McGruff
After such a lengthy discussion of why optimisation might or might not be all that important, it's probably high time to mention some specifics.

Using classes is not harmful efficiency-wise. In fact, when the real optimisation effort begins, at the end of the development process (IMO), it will be much harder to achieve useful optimisations if you don't due to the modularity & testability of a good OOP design.

It is "wasteful" to parse a lot of stuff that isn't used (but you might be surprised how far you have to go before you find a significant loss - try some tests using the getmicrotime fn in the php manual). Anyway, reorganise into separate files and include the ones you need where they are needed.

Same deal with the vars. You might want to define constants instead which are better suited to this task (and immune from variable substitution). Scalars, only of course.

You won't get anything significant out of this (unless one of the vars contains the entire Encyclopedia Britannica), but it is one of those "instinctive" things you do and best practice. I think Roja posted a link in an early post to a site with some benchmark info which is worth a look - other goodies elsewhere on the site as well.

Did you look at any of the sql tips mentioned above?

Got any loops in the script? What's inside them? Other loops? If statements / switch-cases?

I think it might help settle several issues raised here if we were to go through your script in detail - but one bit at a time - and compare execution time after a bit of refactoring.

Posted: Sat Mar 27, 2004 8:53 am
by Buddha443556
McGruff wrote:After such a lengthy discussion of why optimisation might or might not be all that important, it's probably high time to mention some specifics.
Good idea.
McGruff wrote:Using classes is not harmful efficiency-wise. In fact, when the real optimisation effort begins, at the end of the development process (IMO), it will be much harder to achieve useful optimisations if you don't due to the modularity & testability of a good OOP design.
I'll let you get away with this statement so we can get down to details.
McGruff wrote:It is "wasteful" to parse a lot of stuff that isn't used (but you might be surprised how far you have to go before you find a significant loss - try some tests using the getmicrotime fn in the php manual). Anyway, reorganise into separate files and include the ones you need where they are needed.
I agree using [php_man]getmicrotime()[/php_man] is a good idea but try using [php_man]get_included_files()[/php_man] too. Every included file should be contributing to the final page. There's also Apache Benchmark Utility if you have access to it.

One of the biggest bottlenecks is the harddrive and every include() is a hard drive operation. Where as McGruff seems to program in a perfect environment the rest of will have to deal with real world limitations. Program for the worst case situation, a shared server at bofh-isp.com. At bofh-isp.com the hardware is a year old and the server is oversold. Your host is a BOFH (Bastard Operator From Hell) who just waiting for you to use a too much of the server's resources just to charge you extra. Your broke and can't afford that. Can't let McGruff get away with too much proselytizing without a little myself. :twisted:
McGruff wrote:Same deal with the vars. You might want to define constants instead which are better suited to this task (and immune from variable substitution). Scalars, only of course.
Too avoid variable substitution always initialize your variable.
McGruff wrote:You won't get anything significant out of this (unless one of the vars contains the entire Encyclopedia Britannica), but it is one of those "instinctive" things you do and best practice. I think Roja posted a link in an early post to a site with some benchmark info which is worth a look - other goodies elsewhere on the site as well.
The basic idea here is only to load the data that you need.
McGruff wrote:Did you look at any of the sql tips mentioned above?
Databases reside on the hard drive (and even worse could be a remote hard drive) and therefore every database operation is a hard drive operation with some extra cpu usage thrown in for good measure. Query only for the data you need. If use "*", ask yourself if you need all that data? Use only one connection to the database, sockets are not an unlimited resource and can be worse then hard drive operation depending where your connecting them. Avoid mysql_pconnect() on shared server that will get you suspended ( context does matter sometimes ).
McGruff wrote:Got any loops in the script? What's inside them? Other loops? If statements / switch-cases?
What's inside them? That's a good question McGruff. Here's another. What resources are being used in those loops? If it's the hard drive or a socket operation take a closer look.
McGruff wrote:I think it might help settle several issues raised here if we were to go through your script in detail - but one bit at a time - and compare execution time after a bit of refactoring.
I would like that. Maybe I can point out some design consideration that affect efficiency.

Posted: Fri Apr 02, 2004 10:12 am
by llanitedave
Roja wrote:
McGruff wrote:
It's just an attempt to explain an important point: without a well-designed app, it's going to be tough to do any optimising at all.
I disagree with that statement completely - the LESS well designed the app, the MORE benefit you can get from optimizations. If its well designed, its going to be VERY challenging to improve that.
I think McGruff's point was that, while a less well-designed application might NEED more optimization, finding the bottlenecks can be a labor-intensive challenge. A well-designed application will need less optimization, and where it does, the locations of the bottlenecks are more easily found and corrected.

Regarding the "emergence" of an application as opposed to up-front design, I'm kind of fence-straddling on this one. I've built and maintained a major government database application(not in PHP), and it would have benefitted mightily from a better up-front design. Fifteen years later, the design deficiencies have become entrenched, and even where improvements are obviously necessary and technically trivial, the "legacy data" that has been widely distributed simply can't be changed without disrupting the record-keeping of several national laboratories. We're in the same boat as biological evolution having to deal with the blind spot in the backs or our eye-balls: suboptimal design, but it's entrenched and we're stuck with it.

On the other hand, you can do all the up-front design work in the world, and it's not going to help much when half-way through serendipity sets in, and you find that the ingenious design you came up with has some major flaws because of some unknown regulatory requirements, or a simple stupid oversight, or a major change in the needs of the customer, and you have to choose between inserting some quick hack just to make it work, subjecting your project to a significant and expensive delay, or just having to abandon the thing.

Which would YOU pick?

I think good, modular code can help in both these situations -- allowing for good advance design and requirements definition, AND flexibility in dealing with unexpected circumstances.

AND, it will be easier to optimise later, and less likely to need it.

Posted: Fri Sep 24, 2004 10:06 am
by MarK (CZ)
I'm sorry to open old topic but don't think new one is necessary.

When I'm testing the performance, I use only the time of execution as an indicator of code effectivity. Is there a simple way how to measure the amount of memory used by the script? Thx

Posted: Fri Sep 24, 2004 11:43 am
by McGruff
How do you measure time ie are you using Xdebug? If not you might get misleading results. Xdebug can report on op code times - very important since parsing is a large part of the total script execution time.

Imagine a script which just makes native fn calls. This is being compared with a more complex OOP design. The complexity however is all in the design: the OOP script in this hypothetical example is actually shorter since we've managed to eliminate a lot of repetition. Next, you run the simple timer which reports the non-OOP script as the winner. But is it? Once you take parsing time into account things might be quite different. It depends on the script but this can easily account for half the total execution time.

This is where accelerators like Turck etc score. On another day, the complex OOP design might have been the longer of the two scripts: script caching could absorb some overhead.

Not that you're actually likely to find any. Most of the time this is more hypothetical than real. Milliseconds don't matter. I guess everyone has to find out for themselves but code-smithing offers scant rewards, unless you've done something really daft. I do run stuff through the Xdebug profiler and apache ab every so often (the latter is the bottom line) but it's pretty rare you will find anything worth wasting your time on.

http://phplens.com/lens/php-book/optimi ... ng-php.php
http://www.xdebug.org/
http://httpd.apache.org/docs/programs/ab.html

Posted: Fri Sep 24, 2004 12:02 pm
by MarK (CZ)
I'm using a simple method by getting the time in PHP on the beginning (MicroTime()) and count the script time on the end of the file. Is this a misleading method?

And is there a way to measure amount of memory used by script?

Posted: Fri Sep 24, 2004 1:26 pm
by McGruff
Yes it might be - see comments on op codes. You're not taking into account the size of the script and the time it takes to parse. Use Xdebug to profile but apache ab is the real test.

Memory usage: see the first link. In nix, open a couple of consoles. Start "top -d 2" (eg) in one, then run ab in the other.