Here are two examples that should clarify things a bit.
Bob writes a program and makes sure that everywhere he can, he writes the most optimal code he can. Unfurling loops, rewriting libraries, trimming overhead at every possible location.
However, Bob leaves dozens of bug reports lingering because he's too busy writing the most optimal code possible. Worse, fewer and fewer people are using it because their bugs aren't being fixed. In the end, it doesn't even matter because Bob's program runs fine on the majority of hosts, and doesn't suffer from performance problems.
Bob is definitely pre-maturely optimizing.
But now consider Steve. Steve has a set of coding guidelines that he follows because they feel more comfortable to him. Simple things like avoiding large arrays, releasing memory when he's done with it, and heck, even indenting consistently.
Steve doesn't spend a huge amount of time "optimizing", its just part of how he codes. While he may leave some loops curled (costing him in performance), and he may use libraries that aren't perfectly optimized, he's on top of things.
Steve has time to fix bugs, has time to add new features, and even looks at replacement libraries that could improve things.
Is Steve pre-maturely optimizing?
Some would argue that he is. He doesn't "know" that he "needs" to avoid large arrays, and maybe the code would be more maintainable if he didn't. He hasn't commented on how his code performs.. maybe its horribly slow on the majority of systems. Maybe he doesn't even know that it is!
My point is that there is a fine line between good programming practice, and sliding down the slope into premature optimization. Testing certainly plays a big role, and prioritizing does too. Where the right balance is, no one can really say. Its a matter of opinion.
Sadly, I can't claim to be either Steve or Bob. Much like Steve, I don't go out of my way to optimize, but like Bob, I don't seem to have enough time to fix all the bugs in my code. Maybe I'm the wrong person to answer as a result.
