Implementing a shared codebase
Moderator: General Moderators
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Implementing a shared codebase
Hello all,
My boss wants to create a system where potentially hundreds or thousands of sites are running off the same codebase. This doesn't seem like it'd be that hard, as long as most things that are distinct between sites are kept in the database (obviously images and templates will be in a site root). I just bought my own VPS, and I've been playing around with subversion a bit, so I'm wondering if you guys think it would be better to have one shared library between them all, or have each site get its own copy of the library, using svn:externals? (svn:externals basically places external libraries into local folders for use by the main files). Space isn't a concern. With the first, management would be easier, but one error shows up in all sites. With the second, one error will only break one site, but we potentially have many versions of the codebase floating around, which I believe is what my boss wants to avoid. Any thoughts?
Cory
My boss wants to create a system where potentially hundreds or thousands of sites are running off the same codebase. This doesn't seem like it'd be that hard, as long as most things that are distinct between sites are kept in the database (obviously images and templates will be in a site root). I just bought my own VPS, and I've been playing around with subversion a bit, so I'm wondering if you guys think it would be better to have one shared library between them all, or have each site get its own copy of the library, using svn:externals? (svn:externals basically places external libraries into local folders for use by the main files). Space isn't a concern. With the first, management would be easier, but one error shows up in all sites. With the second, one error will only break one site, but we potentially have many versions of the codebase floating around, which I believe is what my boss wants to avoid. Any thoughts?
Cory
Re: Implementing a shared codebase
If I were in your shoes, I'd opt for deploying the shared code with the project as opposed to having the project pointing to a common share. In development, all your projects would be pointing to same shared codebase (Think: SVN trunk), but when releasing to QA or production, you'd run a deployment script (Think: Ant/Phing) that integrates your shared code into the project. When that project goes back into development, you upgrade to the latest shared-source since you'll be testing anyway.
The last thing you want is to update some shared code in production for application A and have application B fail (And you certainly don't want to re-test all your apps just because of a change to the shared source code).
Just my 2 cents though.
Regards,
Marc
P.S. I think your on the right track and I hope your boss doesn't disagree with you. Learn SVN inside and out
The last thing you want is to update some shared code in production for application A and have application B fail (And you certainly don't want to re-test all your apps just because of a change to the shared source code).
Just my 2 cents though.
Regards,
Marc
P.S. I think your on the right track and I hope your boss doesn't disagree with you. Learn SVN inside and out
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Re: Implementing a shared codebase
Well, basically there won't be any "application" separate from the shared codebase. As long as updates work with the online database, there shouldn't be any problems. I think I'm leaning towards what you suggest, but as I said, I think my boss's reasoning for a system like this is to have every website running off the latest codebase (so updates don't involve having to mess with legacy code). Right now, he's got 150 or so sites running a good 40 different legacy versions of his codebase from the last 10 years.
Re: Implementing a shared codebase
Do you work for a web development company that wants to implement a content management system to increase profits by reducing development costs? Regardless, one should expect a project to have a few major releases and several minor releases. Your boss has 150 website, each on 40 different versions. Of the 40 different versions which ones are backwards compatible and which ones are not? If this question can be answered, the problem becomes easier: the next time one of those 150 clients gives your company money, upgrade them to the highest compatible version--if things were done right in the past, I bet you could reduce 40 different versions to under 10. If you can't, you either deal with the legacy code or work for free.
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Re: Implementing a shared codebase
Most of those are not in active development-- and obviously any new clients they take on are using the latest codebase. I'm not sure what you're getting at with your questions-- the codebase IS a CMS, and it's only updated when a new client asks for new functionality. The older the client is, the older the codebase will be. It's not my business, and it's not my code, so I can't speak for their practices. However, I'd like to get this new project under test coverage and version control, and get a good system going for keeping all of the clients up to date.
Re: Implementing a shared codebase
What you are doing is a major project, when you say you're going to have everything contained in one code base.. you need to account for the fact an end user could ask for a custom feature, your boss may not realize how much refactoring would be involved to avoid a "hack-ish" end result. I've been working on something like this for over a year now. My advice to you is show your boss Joomla and start selling sites on that, its easy enough to extend the functionality, uses MVC, you can add new view types and templates
and apply upgrades without overwriting any of your custom plugins. With the available opensource components for Joomla you should be able to make it do anything you need to, or write your own plugins. and still meet the demands of clients.
The only reason to write your own system is get around the licensing restrictions of Joomla, and to allow clients to sign up for your hosted solution and have their "CMS" automatically set up ( on the shared code base ). If you're going to try to automate it ( and compete with me, heh ) by all means, its kept me entertained. Just don't think its going to take 3 months and end up a year later like me still re factoring the basic framework... unless that's really what you want to do.
I'm not saying its not possible, but in practice clients always manage to f**** up your plans.
and apply upgrades without overwriting any of your custom plugins. With the available opensource components for Joomla you should be able to make it do anything you need to, or write your own plugins. and still meet the demands of clients.
The only reason to write your own system is get around the licensing restrictions of Joomla, and to allow clients to sign up for your hosted solution and have their "CMS" automatically set up ( on the shared code base ). If you're going to try to automate it ( and compete with me, heh ) by all means, its kept me entertained. Just don't think its going to take 3 months and end up a year later like me still re factoring the basic framework... unless that's really what you want to do.
I'm not saying its not possible, but in practice clients always manage to f**** up your plans.
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Re: Implementing a shared codebase
This project is definitely all about automation. We want to abstract out everything we do similarly across most of our projects: creating the database, installing the codebase, adding users, adding sample content, running unit and integration tests to make sure everything works well. I agree it will be tricky to design the custom code on top of the codebase, but honestly, 90% of our clients don't need anything more complex than a one- or two-level page management system. These are simple content websites, not web applications. The majority of custom work will be on the front end. If they need custom code, it will be simple enough to switch out classes. That's the benefit of well-made OO architecture... you can swap out components easily.
I'd rather not use an open-source CMS. They tend to be overly large, and try and please everyone. Not to mention, I generally don't agree with a lot of the design decisions the open source CMSs and frameworks have made.
I'd rather not use an open-source CMS. They tend to be overly large, and try and please everyone. Not to mention, I generally don't agree with a lot of the design decisions the open source CMSs and frameworks have made.
Re: Implementing a shared codebase
I shared a similar thought process when I started working on my version of this idea. What I did was I took all code I had ever written on all projects and merged into one working application, then I created a directory to house all the "common" code, as I added a new client I created a new directory for that client, outside of this "common" folder, using rewrite rules and conditional apache rewrites I am able to setup a "primary" and secondary document roots ( or how many every additional i need ), if the file exists in the primary document root the URL is mapped to that file, this way I could "overload" images and css scripts that were in the document root. Also my old legacy code didn't have a centralized front end controller yet so it was necessary.
The problem with this setup is, in theory it is "perfect", but you're still very prone to duplicating code. Yes OOP lets you switch out objects, but if I'm doing something custom for one client, I want to be able to offer it to the rest of the clients, I quickly abandoned the multiple directory scheme and adopted front end controllers needless to say
The problem with this setup is, in theory it is "perfect", but you're still very prone to duplicating code. Yes OOP lets you switch out objects, but if I'm doing something custom for one client, I want to be able to offer it to the rest of the clients, I quickly abandoned the multiple directory scheme and adopted front end controllers needless to say
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Re: Implementing a shared codebase
A little code duplication is ok with me. After the project's done, we can look at refactoring out the duplicated code so that other clients can take advantage of the same functionality.
What's wrong with having "front end controllers" and multiple directories?jshpro2 wrote:I quickly abandoned the multiple directory scheme and adopted front end controllers needless to say
Re: Implementing a shared codebase
Well nothing, I guess I'm looking at it from a different point of view from you. I am looking at it from a business standpoint not a technical one, I personally try to do only write code that I will be able to go out and sell over and over, to multiple clients.. My time is money and I only purchase assets, I choose not to work for money I choose to let my money work for me. If you design the application good from the start I guess you won't have any problems though no matter how you approach it, technically speaking.. godaddy for example is a "factory" of VPS hosting, they auto provision the servers through a 3rd party API and collect a reseller fee. If you call godaddy to ask for a custom site they will probably not do business with you.. I'm thinking more of a hosted solution like that
see http://www.netsuite.com
Their anual contracts start at $80,000 per year, I had a client that was trying to build a "clone" site of
http://www.csnstores.com
he was looking at going onto netsuite.. Theres lots of people offering doing the same thing, and often as developers we like to think if we design something using good backend code a project will succeed business wise, which holds true to a degree, but in the end most clients don't care about the code, all that matters is that you save them time.. which they see as money, and in result will give more money to you
see http://www.netsuite.com
Their anual contracts start at $80,000 per year, I had a client that was trying to build a "clone" site of
http://www.csnstores.com
he was looking at going onto netsuite.. Theres lots of people offering doing the same thing, and often as developers we like to think if we design something using good backend code a project will succeed business wise, which holds true to a degree, but in the end most clients don't care about the code, all that matters is that you save them time.. which they see as money, and in result will give more money to you
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Re: Implementing a shared codebase
Well, that's the point of the shared codebase... code than can be used over and over again by multiple clients.jshpro2 wrote:I personally try to do only write code that I will be able to go out and sell over and over, to multiple clients..
To be honest, I think the business side is my boss's strongsuit. What I'd like to do, is revamp how his business does coding. Everything that can be automated, will be. Eventually/ideally, the only coding we'd be doing is custom work. Which is what I enjoy, far more than building a "one size fits all" solution. The more we do that, and the more we refactor the custom code into the codebase, the less he'll have to offload to the other company's developers. In the end, the client gets a much faster turnaround, and also has the extent of our codebase at their fingertips, on a pay-per-feature basis, saving them "time... which they see as money, and in result will give more money to [us]".jshpro2 wrote:Theres lots of people offering doing the same thing, and often as developers we like to think if we design something using good backend code a project will succeed business wise, which holds true to a degree, but in the end most clients don't care about the code, all that matters is that you save them time.. which they see as money, and in result will give more money to you
Re: Implementing a shared codebase
Yeah I feel you, im just saying be careful to not slip into the mindset of writing a "client A" module and instead try to write a "generic solution A" module, I realized personally there was no need to have any special case scenarios and that everything could be in 1 codebase, not just partially in that codebase