Page 1 of 2
Conditional includes or...
Posted: Mon Jan 14, 2008 3:35 pm
by Ambush Commander
I am attempting to make a few changes to the plugin structure of some of my code, and I'm wondering how I should handle these things to be the most efficient when used in conjunction with bytecode caches.
Unfortunately, the only things I know about this mythical beasts are third-hand quotes and vague precepts. For example, autoload and conditional includes are bad for bytecode caches. On the other hand, including large amounts of code that end up not being used is not very nice either.
So guys, I'd like a number. How many extra kilobytes does it have to be before you bite the bullet and use a conditional include/autoload to save the effort of having to include everything?
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 3:42 pm
by alex.barylski
Have you heard any reason as to why autoload would be bad for bytecode cache?
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 3:52 pm
by Ambush Commander
This URL:
http://www.pooteeweet.org/blog/538
Basically, autoload needs to be evaluated at runtime, so the cache can't fully form the classes until then (it can cache the ops, but they are not necessarily the most efficient).
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 4:18 pm
by Ambush Commander
I've just thought of a curious way of solving this file: dynamically generating a PHP file with the necessary includes based on a user configuration. You get the benefits of an opcode cache, you can use plain old require with impunity (since everything is centralized, although it would probably be a good idea to remove all of the require_once's littered in my code), and only necessary code is used.
The downside is considerably increased complexity, but I believe that most compiled languages work similarly (if you don't need a module, it doesn't get linked in).
Thoughts?
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 5:54 pm
by georgeoc
It sounds like a sensible solution to me. Kind of like an advanced cache system - if the user configuration changes you need to regenerate the file. It should give a nice performance increase with less dynamic, bytecode cache-friendly code.
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 5:59 pm
by Weirdan
Thoughts?
Well, if you know in advance what files will be needed this is a way to go. Autoload is useful when you don't know what actually should be included - basically it's the same scenario where you would use conditional includes.
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 7:20 pm
by Ambush Commander
And here I was thinking that I had thought up of a novel technique!

)
Has anyone ever seen something like this in use in practice?
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 7:41 pm
by Weirdan
Usually people take this a step further and compile all the files into one large file. This helps to reduce number of fstat's (which are resource expensive on most filesystems) and you're already forced to compile the list of includes in a particular order - so why not include their contents directly instead?
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 7:53 pm
by Ambush Commander
Ehh... that's what I'm already doing! (spins around)
Well, imperfectly, anyway. The user isn't expected to do the compilation, so anything I produce has to be one-size-fits-all. This means a number of extra PHP files that still need to be dragged around and included if necessary (greatly reduced though, maybe about 5). In fact, all of this conditional include/loading stub business was me trying to deal with this messiness that came about while I was compiling the single PHP file.
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 9:18 pm
by Christopher
Ambush Commander wrote:I am attempting to make a few changes to the plugin structure of some of my code, and I'm wondering how I should handle these things to be the most efficient when used in conjunction with bytecode caches.
I hate to ask the question -- but do you have a real performance problem to solve here?
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 9:36 pm
by Ambush Commander
Maybe,
this was the picture that first convinced me there was something wrong with the way I was doing includes.
Although, I see your point: there aren't that many plugins, so it's not that big of a problem yet (they represent about 15 files). But I've got some doing it one way and others doing it another way I'd like to get some consistency hammered down.
The performance problem might not be here. Hmm... (The fact that I'm reading stuff by Knuth right now might not be helping either

)
Re: Conditional includes or...
Posted: Mon Jan 14, 2008 10:24 pm
by wei
May be profile the application first before considering any performance improvements.
Re: Conditional includes or...
Posted: Tue Jan 15, 2008 4:49 am
by Maugrim_The_Reaper
APC caches autoloaded files. It's not quite as fast as full caching but I don't think the difference is significant. The blog post linked to was taken as gospel before before folk realised any caching was better than no caching at all. Folk also missed any number of factors about including files which could hit performance even more than autoload.
Here's an interesting benchmark post from Greg Beaver:
http://marc.info/?l=pear-dev&m=119058592212884&w=2
Re: Conditional includes or...
Posted: Tue Jan 15, 2008 1:24 pm
by Christopher
Did you actually read that post. He talks and talks and talks ... and finally talks himself into going the direction is wanted to go anyway.

The performance benchmarks show insignificant differences in that test, except that __autoload was much slower in one test. That's not exactly the result that gets posted in the summaries. It is also hard to know what effect the code changes he made had on performance -- he refactored the include_once code to the other two styles and may have done a little tuning in the process.
He is trying to solve a problem for PEAR2 and has talked himself into a direction. But his arguments really come down to your opinion about using include_path and how to deal with newbie mistakes.
Re: Conditional includes or...
Posted: Tue Jan 15, 2008 2:25 pm
by Maugrim_The_Reaper
I actually read that post - and the 100+ related ones before and after it

. The idea is quite simple. You offer easy option for the majority of users (include some file and use) with another option (overload the loading method) for advanced users. Let's say I rewrote PEAR2_Autoload to use an absolute path over a relative path, or bundled all classes into one file? Can't do that easily if a library is littered with require_once() everywhere... In the past I've used Phing to reorder stuff this way...
Is it a question of performance or developer maintenance? Personally I don't see huge maintenance potential by using require_once. I can write or read a large library without them if the code is documented well, and class references follow the PEAR convention. So it's largely a performance question. Autoload can outperform require_once (lazy file includes) unless you do the apc.stat=0 trick in which case it heads downhill fast. The differences are considered even more severe on a multi-processor platform (I think Gopel had the range from 2% (single) to 10% (multi) of a difference). The difference is also severe on large packages with multiple require_once redundant calls (i.e requiring 50 files when only 15 are used), something the Zend Framework is really good at

.
At the end of the day, whether you see one of Greg's isolated posts as self-persuasion or not the mounting rhetoric interpreted from half-understood statements and APC bugs in 2005 against autoload melted last Autumn after the long drawn out PEAR debate.