Code Reuse

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Code Reuse

Post by Ollie Saunders »

Start by reading this blog post questioning the point in code reuse.

<bigFatRant>

My job has led me to be involved in the creation of numerous small websites. I now have loads of code scattered about on 5 different machines and despite my strongest skills, wills and desires I have completely failed to benefit from code reuse, in fact I have suffered a detrimental effect from my attempts. On a couple of occasions I've chosen to write again, the same piece of code, I wrote less than week ago, because the context was slightly different and it would take me ages to dig out that old bit of code. Even if I had found that bit of code then I'd only be copying and pasting it anyway, violating DRY. What's the point? Perhaps I shouldn't consider myself a fool for doing this any longer. Perhaps DRY is a lie.

Subversion is certainly an absolute crock. Seriously people, why are you using that?! It's rubbish for what it's intended for. The only thing it is good for is what we did for the throw down, which is keeping remote workers synced in a rapidly updating small environment where programmers can easily inform each other of their changes.

I spent ages devising this scheme for how I'd use it with branches, tags, externals definitions etc. etc. I had this whole 600 word guideline document set out with how I was going to manage it all. It was marvellous.... in theory. The minute I started to use it I realised it was totally unworkable! One little mistake where I type the wrong thing in an external would require something like 7 steps to undo, waiting after most of them. You can't afford to make mistakes because they are so complex to undo, you fear making them. And then you get confused and your whole code is broken and scattered all over the place. The experience is emotionally draining and distressing. Subversion is also useless as a deployment tool - the second you get a conflict (which is easy to do with externals) on your production server your whole site is down, the PHP code immediately invalidated by Subversion's diff markup, until you fix it, that's bloody great that is.

Granted, I think my situation is unique; I'm trying to achieve an environment where you can share code across multiple projects, clearly, Subversion wasn't up to the job. I'm interested in trying GIT. The author (some of you might know him) feels as strongly as I do, as this youtube video will show. His unclouded non-conformist nature is what I needed to release that Subversion is rubbish.

Whilst I've learnt how to write good object oriented code that is loosely coupled, embeddable, generalised, unit tested and all that jazz - it takes <span style='color:blue' title='I&#39;m naughty, are you naughty?'>smurf</span> ages to do it! Frameworks and libraries are the one area where it does pay off to put all that effort in. Why? Because the whole world can benefit. For the purpose of small company it's pretty pointless and big ones are full of too many idiots to actually do a good job, they should both stop, they'll save a lot of money. I always wondered people I considered lesser programmers than myself were able to get more done, it's because they weren't on this mad goose chase I was.

Bleak, isn't it? Yes it is but there is hope. What can we do?
  • Stop worrying about that website you haven't updated to use the latest version of this or that. Web code doesn't have much use beyond 2 years usually anyway.
  • Consider meta programming. I had this idea to create a website where you could download customised class hierarchies. Before download you would choose what functionality you wanted. So you could turn off the stuff you don't need and benefit from the simplicity. That kind of thing is going to be really useful for making libraries more accessible and contextual.
  • Packaging, Package Management, Dependency Resolution, Behaviour Driven Development (evolution of TDD), Aspect Oriented Programming, combined will go a long way to make libraries easier to manage and more get-at-able.
  • More dynamic languages. You guys really don't know what your missing until you've looked at modules and mixins in Ruby. I'm not completely sold on Duck Typing just yet though but we'll see.
  • Domain specific languages. There's almost no desire to reuse code here. Think about regular expressions and XSLT. The only time you ever reuse code here was when someone was doing something you couldn't do yourself. Actually they mean that people have to know more languages to get things done, this leads to more rounded programmers.
Unless you are contributing to libraries or languages your just creating noise. You'll increase the value of your company's core code base infinity fold by releasing it to the public and getting it generalized for wide spread use. Of course there's value in small scale code reuse, perhaps you'll have on or two include files with a bunch of functions that you commonly use but it's only a small benefit anyway, you can't really scale it up.

We're probably 5 to 7 years away from getting all this sorted. Till then don't stress yourself out, like I did, trying to achieve the impossible.

</bigFatAunt> :P
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

Wow! Those were some brash comments you made there.

I think your experiencing the troubles of working in complex environments as a sole developer. SVN is still handy if you like the security of perpetual backups.

As for code reuse. Frameworks, libraries, etc are usually the impetus behind code reuse. Perhaps your expecting to much code reuse???

Application specific code is going to be quite difficult to reuse.

Maybe you can post your code for these web sites and get a second or third opinion?
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

ole, I think you make a number of very practical observations about the reality of web development. It also sounds like you might be overthinking the problem a little. ;)
(#10850)
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Post by Luke »

The minute I started to use it I realised it was totally unworkable! One little mistake where I type the wrong thing in an external would require something like 7 steps to undo, waiting after most of them. You can't afford to make mistakes because they are so complex to undo, you fear making them. And then you get confused and your whole code is broken and scattered all over the place. The experience is emotionally draining and distressing. Subversion is also useless as a deployment tool - the second you get a conflict (which is easy to do with externals) on your production server your whole site is down, the PHP code immediately invalidated by Subversion's diff markup, until you fix it, that's bloody great that is.
I've been having a love/hate with subversion since I began using it. I kind of assumed that the troubles I'm having with it are due to my own ignorance, but none the less I've suffered many hours of fixing a very easy-to-make mistake in subversion. It has been very frustrating, but the benefits of using it are immense. When it's working properly I really do wonder how I was developing without it. Especially now that I have another developer here. It makes working on the same project so much easier.
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

Hockey wrote:Wow! Those were some brash comments you made there.
Yeah I think I needed to get it out of my system.
I think your experiencing the troubles of working in complex environments as a sole developer. SVN is still handy if you like the security of perpetual backups.
Yes, you're right. On a single project Subversion is a problem, if you are working on one then there isn't that issue. I think basically I need to think about getting a position where I can be writing web applications instead of countless websites.
As for code reuse. Frameworks, libraries, etc are usually the impetus behind code reuse. Perhaps your expecting to much code reuse???
I almost certainly was. My initial post was largely and admission of that fact.
Application specific code is going to be quite difficult to reuse. Maybe you can post your code for these web sites and get a second or third opinion?
That sounds like a lot of hard work.
aborint wrote:ole, I think you make a number of very practical observations about the reality of web development. It also sounds like you might be overthinking the problem a little.
What makes you say I'm over-thinking it?
I've been having a love/hate with subversion since I began using it. I kind of assumed that the troubles I'm having with it are due to my own ignorance, but none the less I've suffered many hours of fixing a very easy-to-make mistake in subversion. It has been very frustrating, but the benefits of using it are immense. When it's working properly I really do wonder how I was developing without it. Especially now that I have another developer here. It makes working on the same project so much easier.
I assumed ignorance also, but after learning about the more advanced features and consulting experienced developers I'm pretty confident I'm no longer ignorant. When I asked those developers they simply suggested doing things that I was doing already. I realised that Subversion is flawed. It's not useless, in some environments it's excellent, I can't really speak for multiple developers, but it could be much better and I'm now really interested in GIT as an alternative.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

The problem with Subversion is that's it's commonly available ;). I know subversion has problems but a lot of people know it, and familiarity is only ever overturned when something more compelling turns up (GIT is not yet compelling enough with its limited mindshare). Maybe it's a factor that few people use Subversion's advanced features a lot? For simple copying, branching and distributed work it's still mightily useful. Most people don't need, or want, to look further.

The article states one obvious facet of re-use - re-use demands generality. It's extremely difficult and expensive to write highly complex and involved code which is general enough to be reusable elsewhere. Just as its difficult to predict where it could be reused. How many man hours went into PHP's frameworks, libraries and similar? It would cost an absolute fortune! How much do you think you would really be willing to pay for something like Swiftmailer, or a high value library like HTMLPurifier? Or a massive framework like ZF or Symfony? Reuse is expensive which is why it's usually far more common in open source where monetary expense is not always an issue.
Whilst I've learnt how to write good object oriented code that is loosely coupled, embeddable, generalised, unit tested and all that jazz - it takes smurf ages to do it!
The cost of reuse ;). Also the cost of maintenance. The thing is all those companies are reusing their code across their internal projects, so they are reaping benefits of a sort. They reap more if they can find/contribute to a matching open source input. It's easy to skip testing and refactoring. How much time do you currently spend maintaining (not adding new stuff to) your expensive tesable refactored code? ;) I'd bet if you dropped those practices you'd spend a lot more.

On your points - here's a pretty humourous tale. RSpec (that Ruby BDD library) liked the Ruby way so much that the specs are not classes (but those other eval'd things from Ruby). It all looks cool, but since they're not classes, they can't be extended. If they can't be extended they can't be changed, reused or retrofitted. In fact you have go dig around all the other poorly documented classes (Rubyists don't need to comment you see :)) to figure out how to change anything. So the poster child for BDD is not all that pretty underneath. BDD is still cool though ;). RSpec has been such a moving target since BDD started up it's a wonder it still has so little code anyway.

Totally want AOP!
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

The problem with Subversion is that's it's commonly available Wink. I know subversion has problems but a lot of people know it, and familiarity is only ever overturned when something more compelling turns up (GIT is not yet compelling enough with its limited mindshare). Maybe it's a factor that few people use Subversion's advanced features a lot? For simple copying, branching and distributed work it's still mightily useful. Most people don't need, or want, to look further.
If Subversion works for the task at hand I have no concerns there. I tried to get git up and running today and had a few difficulties with different versions and portability, so for now I'm exploring Bazaar which seems to be as easy to use as subversion (probably easier) and does all this extra stuff on top.
Reuse is expensive which is why it's usually far more common in open source where monetary expense is not always an issue.
I think this really highlights why open source development is going to lead the way for software. In the not too distant future you probably won't be able to manage without open source tools and you won't be able to keep up by making propriety alternatives. It's an exciting time.
The cost of reuse Wink. Also the cost of maintenance. The thing is all those companies are reusing their code across their internal projects, so they are reaping benefits of a sort. They reap more if they can find/contribute to a matching open source input. It's easy to skip testing and refactoring. How much time do you currently spend maintaining (not adding new stuff to) your expensive tesable refactored code? Wink I'd bet if you dropped those practices you'd spend a lot more.
That's a good point. Perhaps I should say: Yes to DRY principle within a project but when you start writing your own frameworks and trying to generalize across multiple projects and environments then you might as well be using/creating/contributing to a library instead. There's no way such things can be created without loads and loads of effort.
RSpec (that Ruby BDD library) liked the Ruby way so much that the specs are not classes (but those other eval'd things from Ruby).
It wasn't originally. The author gives this talk where he's talking about a "should" method returning an object. And don't think that using closures over classes is necessarily the Ruby way, it's not. I would guess his motivation was to make it so that you could achieve really high readability for specs...
It all looks cool, but since they're not classes, they can't be extended.
...and so there is a cost. It'll be interesting to see if it was worth it. I have to admit I'm with you on this one it does sound pretty unworkable for the future.
Rubyists don't need to comment you see
Rubyists probably need to comment as much as PHP programmers or at least those using the language in a more dynamic way will. You always need to write manuals/documentation. Even if you had a compiler that understood pure English it would be necessary to provide a non-executable overview of the copy.
Totally want AOP!
You might have guessed, I've been watching quite a few Google Tech Talks recently, here's one on AOP. For those who are unfamiliar with AOP, you'll benefit from reading up a little and familiarizing yourself with the concepts, this talk demands serious concentration but is exceptionally rewarding.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Rubyists probably need to comment as much as PHP programmers or at least those using the language in a more dynamic way will. You always need to write manuals/documentation. Even if you had a compiler that understood pure English it would be necessary to provide a non-executable overview of the copy.
In case you missed the sarcasm ;).
The RDoc output for rspec is absolutely horrible - which seems oddly common in Ruby. PHP's commenting obsession is such a thing to miss.
And don't think that using closures over classes is necessarily the Ruby way, it's not. I would guess his motivation was to make it so that you could achieve really high readability for specs...
I remember discussing PHPSpec with someone recently, and they requested I not mention rspec too much since PHP programmers would get upset about me porting a Rubyism into PHP and dismiss it right then and there ;). I agree they're completely readable - but it's still pushed extensions into an arcane art until you figure out how the runner classes operate and collect behaviours. Dave Astels was quoted at some point as keeping classes as a possible change for the future since it was a common complaint - not sure if that will happen in reality.

Google Tech Talks are usually pretty good - Glad to meet another viewer!
Selkirk
Forum Commoner
Posts: 41
Joined: Sat Aug 23, 2003 10:55 am
Location: Michigan

Post by Selkirk »

I agree, Subversion is a terrible tool for deployment. So, don't use it for deployment. Try something like the pear installer.

One big problem that I've seen before is trying to put too much under SCC. For example, trying to manage all of your libraries and tool dependencies with SCC. Personally, I never use externals, nor do I check in code libraries that I haven't written. Been there, done that, bad news.

I agree that some things lend themselves to reuse and some don't. You can divide a system up into multiple categories based on degree of reuse.
  • Application - Code useful for one application
  • Business - Code useful for one industry or company
  • Architectural - Code useful for one implementation architecture
  • Foundation - Code useful for all businesses and architectures
Foundation code is very reusable, while application domain code is not. SwiftMailer and HTML Purifier are great candidates for reuse because there are difficult issues in both problem domains that benefit from using a standardized, well tested library. They are examples in the foundation. That doesn't mean that it would pay off for you to create your own version of such a library. If you want to be productive, you should be acquiring most of this code from third parties.

Architectural code is where the action is in PHP right now. This code is moderately reusable. The frameworks all fall into this category.

Business code is much less reusable, but still can be reused. Do you write a brand new user authentication and user management code for each project, or do you reuse across projects? If you copied and pasted that code, or you want to add a feature across projects or fix a bug, it requires more work for more copies.

Application code is hardly reusable at all. Don't try to abstract and generalize this code.

Robert martin has a lot of writing on the relationship between deployment, dependencies and reuse in his book "Agile Software Development."

Here are some principles relating to deployment:
  • The Release-Reuse Equivalency Principle - The granule of reuse is the granule of release
  • The Stable Dependencies Principle - Depend in the direction of stability
  • The Stable Abstractions Principle - A package should be as abstract as it is stable.
  • The Common Closure Principle - The classes in a package should be closed together against the same kinds of changes. A change that affects a closed package affects all the classes in that package and no other packages.
  • The Common Reuse Principle - The classes in a package are reused together. If you reuse one of the classes in a package, you reuse them all.
  • The acyclic dependencies principle - Allow no cycles in the package dependency graph
I totally agree that OO code can be expensive to write. However, I find the big payoff not to be in development time, but in reliability. For me, OO code is easier to test and maintain. I think many people become disillusioned with OO code because its sold on the basis of improving development times. It can only do that for certain kinds of problems. I find that many use OO as an excuse to overbuild and over generalize, for example, declaring an interface where there is only one class that implements it. One of the biggest traps seems to be database abstraction and ORM.
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Post by Luke »

One of the biggest traps seems to be database abstraction and ORM.
What do you mean by that?
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

ole wrote:What makes you say I'm over-thinking it?
Because you are complaining about problems most people accept but that zealots argue about. ;)

I agree with you about Subversion. I tend to just use CVS because it is built into more things (like Eclipse). The problem is not the tools themselves, but the terrible interface to them. GIT looks interesting. There is a simple web interface for it in PHP that tried a while back.

As to your reuse problems, it is not clear whether you are any different that me -- except that you have higher expectations. I do get a fair about of reuse from my codebases, but that is more form getting similar problems to solve. Continuous upgrading is still a problem though.
(#10850)
sike
Forum Commoner
Posts: 84
Joined: Wed Aug 02, 2006 8:33 am

Post by sike »

Selkirk wrote:I agree, Subversion is a terrible tool for deployment. So, don't use it for deployment. Try something like the pear installer.
this is not a problem with subversion but rather a problem of a bad deploy strategy? i would consider svn update on a live system a first step to heart problems (; there are working methods for using svn for deploying but you they need planing and organization (e.g. release management).

reusable code comes at high costs so i try to generalize as late as possible. i have done some projects years ago where i tried to build a framework on the fly and guess what? i failed miserably even to the point of risking failure of the whole project.
my current strategy is to get the project done and generalize only parts of my code base if i see similarities from previous projects (aka harvested frameworks). this works out ok thus far but i have to admit that the resulting "framework" is more of a library of building blocks for most of the projects i do.

cheers
Chris
User avatar
Benjamin
Site Administrator
Posts: 6935
Joined: Sun May 19, 2002 10:24 pm

Post by Benjamin »

SVN Works fine for me, alone and with teams. Not sure what problems or mistakes you guys are referring to.
Selkirk
Forum Commoner
Posts: 41
Joined: Sat Aug 23, 2003 10:55 am
Location: Michigan

Post by Selkirk »

NSG, I just mean that it is possible to spend a great deal of effort abstracting away the database, but what do you actually gain? For anything non-trivial, you're still going to have to delve into SQL. Most projects will never switch databases. I remain extremely skeptical of DBA layers & ORM.
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Post by Luke »

yea I agree for the most part. it is pretty uncommon to switch databases in an app. I've never had to switch databases in any project before.
Post Reply