TDD workshop. Anybody welcome; Data storage abstraction.

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

Post Reply
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

As an education experience we're going to run a (almost) live TDD session here on the forum. The end-product we're looking to produce is an abstraction library for reading and writing data formats. We'll set out to support at least three formats initially and then see how many we can support beyond that.

Requirements

Create a library for reading and writing tree-like data structures in the simplest way possible. The library should be able to read data in one format and write it back out in a different format.

Permitted PHP versions:

PHP >= 5.2 with any extensions we deem required. We may opt to include some 3rd-party code if we see fit.

Supported formats:
  • XML
  • YAML
  • Basic PHP Array
Features:
  • Load an existing configuration/data store in any of the aforementioned formats
  • Create a new configuration/data store in any of the aforementioned formats
  • Read a value with a known key from the data store
  • Allow nesting of keys (i.e. a hierarchical structure like XML)
  • Set or change a value
  • Delete a value
  • Commit/save changes to a loaded data store.
Don't forget!
  • TDD does NOT permit you to implement a feature without first writing a test to declare what you're aiming to acheive
  • Test methods should be short and concise
  • You CAN and SHOULD discuss a planned API before going ahead and using it. Talking about the API is NOT the same as implementing it. We have to communicate with each other otherwise we'll have have different ideas about how it will work. This is particularly important because we're not all sitting with each other
We'll be using SimpleTest to do our unit testing. If you don't have a copy, download it now and have a little read over the documentation.

http://www.sourceforge.net/projects/simpletest

How to participate

Introduce yourself and express your interest, along with telling us how much you already know about TDD. Anybody is welcome to participate; just let us know who's working with us since we'll effectively be working in a team-oriented situation.

If you are posting code, first tell us what you're very roughly thinking at an interface level (NOT what you plan to implement). Then get some feedback since we may think it can be improved, or we may like what is planned. If we agree to head that direction, your code should be posted here showing the test first, then the code you wrote second.

Example of how NOT to post what you are thinking:

Code: Select all

$value = $store->get($wantedKey);
 
// .. snip ...
 
//Look up a value
foreach ($dataValues as $key => $value) {
  if ($wantedKey == $key) {
    return $value;
  }
}
That is showing us too much about the implementation. Everything up until "... snip ..." is all we care about.

I'll finish my lunch and then get the ball rolling by asking people to suggest in interface for creating a new data store then setting a value in it :)
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

Okie doke, so we need to spend some time discussing an interface before we write any code. Let's start by thinking about how we'd set a value in the data store and reference it by a known key before saving it.

My $0.02, it can probably be expressed better I'm sure. Fire away :)

Code: Select all

$store = new DataStore(new XmlBridge("/path/to/file.xml"));
$store->set("/a/know/key", "my value");
$store->save();
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by matthijs »

Great! I'm looking forward a lot to this exercise. Have a little bit of TDD experience, but not much.So don't go too fast. Better take small steps so we can think about each one carefully.

The XMLBridge: sounds like the one responsible for writing to and reading to XML. Seems fine.

Code: Select all

$store = new DataStore(new XmlBridge("/path/to/file.xml"));
I think I know why you did it like this, but still I'd like to know the idea behind it. Is it to aggregate/compose the different classes instead of inheriting a base read/write class or duplicating methods in them? (may have the terms not quite correct)

Besides that, I think ->set() and ->save(0 are perfectly clear.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

I'd favour composition over inheritance. Having the DataStore understand how to use some basic read-write API without caring about what's going on under the surface just seems like a good idea.

It may be the case that we don't need the DataStore class and we can just directly instantiate an "XmlStore" or something, but the abstraction just seems clearer expressed how I've written it. This is basically a strategy (or maybe adaptor) pattern which fits with what we're talking about. Writing XML, YAML etc are different strategies for the same general purpose. I'm not thinking about the way that strategy will be implemented right now though. Thinking in Design patterns is just something which ends up coming naturally to me; it doesn't mean I've gone down the right path.

The good thing with TDD is that you don't have to get it right the first time around. You need to get it roughly right however. The adjustments can happen later (refactoring ... a vital step in TDD) and the tests will help massively whilst making such adjustments.

We'll take it slowly yes. Realistically because we're all at work and doing other things this is going to take us at least a few weeks to complete. I have no problem at all taking time out regularly to explain what's going on to others.
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by matthijs »

Excellent. Your explanation makes sense. It is the same pattern as is often used in validation, in which a Validator composes different Rules. So you can have many Rules and the Validator only cares about a few methods.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

Ok, so our interface for the StorageBridge (name sound ok?) is starting to look like this:

Code: Select all

/**
 * A means of reading/writing data to a storage backend.
 */
interface StorageBridge {
  /**
   * Store a value using $key to reference it.
   * @param string $key
   * @param string $value
   */
  public function set($key, $value);
  
  /**
   * Persist changes.
   */
  public function save();
}
"StorageBackend" sounds better actually. Implementing it as XmlBackend, YamlBackend etc.
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by matthijs »

Actually, StorageBridge sounds confusing to me. It sorts of combines storage with an action. DataStore seems clearer to me. A datastore is something you can put data in and get it back out. Simple enough.

This is my first impression. Maybe I'll have to give it some more thinking.

Or is it that you start with StorageBridge as an interface to implement that for the specific Bridges, like XMLBridge, CSVBridge, etc? Maybe it's something about Bridge that I don't like. I know it should be called something that relates to Reading and Writing, I can't think of something else right away.

Maybe some others could shed their thoughts on this.

The code seems clear enough.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

"StorageModel"
User avatar
arjan.top
Forum Contributor
Posts: 305
Joined: Sun Oct 14, 2007 4:36 am
Location: Hoče, Slovenia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by arjan.top »

I have some experience with TDD ...

I am currently in school so I will look at posts when I am home, I have scanned them and it looks good :)
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

arjan.top wrote:I have some experience with TDD ...

I am currently in school so I will look at posts when I am home, I have scanned them and it looks good :)
Good good :)
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by matthijs »

Chris Corbyn wrote:"StorageModel"
Seems better to me. Then you'd get a XMLModel, YamlModel, CVSModel, etc, isn't it? That would make sense, because with the average "Model", like UserModel, BlogpostModel, etc you also have similar operations like setting and getting values, saving them, etc
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

Just to give a sense of direction with regards to the directory structure I've attached a ZIP file containing a dummy test.

I personally don't use the built in test suite, but for the sake of not confusing everyone, I've laid out the base directory structure as follows.

Code: Select all

/
/classes/ <--- all classes we implement go here
/tests/
/tests/configure.php <-- A script to include the required parts of SimpleTest
/tests/AllTests.php <-- A test suite for grouping all tests together
/tests/unit/ <-- A place to put each individual test
/tests/lib/simpletest <-- A copy of SimpleTest
 
Download it and have a poke around.  I've made use of SimpleTest's autorun.php feature so if you open your web browser and point it to:
 
http://localhost/data_store/tests/AllTests.php
 
Then you should see the tests run.  It will fail because I added a DummyTest for demonstration purposes.  I'll delete that when we begin though.  The DummyTest instills a bit of confidence that we have SimpleTest set up correctly.
 
If you point your browser at:
 
http://localhost/data_store/tests/unit/DummyTest.php
 
Then you are running that single test by itself.  Again, it will fail as expected.
 
If you've never used SimpleTest then make this test pass and watch what happens when you run those two files again.  If you have used SimpleTest then you can skip this since it's pretty obvious ;)
 
NOTE: Because autorun.php is included there's no need to call $test->run( ... ).  You can also run the tests on command line.
 
If I run the two test cases (the group one and the single one) it looks like this:
 

Code: Select all

w3style-2:data_store d11wtq$ php tests/AllTests.php 
AllTests.php
1) Expected false, got [Boolean: true] at [/Users/d11wtq/data_store/tests/unit/DummyTest.php line 7]
    in testNothing
    in DummyTest
    in /Users/d11wtq/data_store/tests/unit/DummyTest.php
    in All DataStore tests
FAILURES!!!
Test cases run: 1/1, Passes: 0, Failures: 1, Exceptions: 0
w3style-2:data_store d11wtq$ 
w3style-2:data_store d11wtq$ #Just me hitting enter a few times to make a gap ;)
w3style-2:data_store d11wtq$ 
w3style-2:data_store d11wtq$ php tests/unit/DummyTest.php 
DummyTest.php
1) Expected false, got [Boolean: true] at [/Users/d11wtq/data_store/tests/unit/DummyTest.php line 7]
    in testNothing
    in DummyTest
FAILURES!!!
Test cases run: 1/1, Passes: 0, Failures: 1, Exceptions: 0
w3style-2:data_store d11wtq$
PS: The ZIP has all the docs and other unneeded parts of SimpleTest removed just to make it smaller to attach here ;)
Attachments
data_store.zip
(135.19 KiB) Downloaded 403 times
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

matthijs wrote:
Chris Corbyn wrote:"StorageModel"
Seems better to me. Then you'd get a XMLModel, YamlModel, CVSModel, etc, isn't it? That would make sense, because with the average "Model", like UserModel, BlogpostModel, etc you also have similar operations like setting and getting values, saving them, etc
Agreed. Let's make a start :) Just waiting to make sure people are comfortable with that dummy layout I attached before we begin.
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by John Cartwright »

Hey Chris, sorry for jumping in the middle.

You mentioned you prefer composition over inheritance (I do as well), but as we recently discovered while creating the configuration component of the skeleton framework, is how to deal with errors the native php functions might spew. Considering the use of error suppression is a no-no, I am interested to see how these are handled. After some discussion, we thought that it might be better if the child class extends the parent class, but the twist is the base class holds our public interface. The base class would then temporarily set the error handler to a method in the base class.

I.e.,

Code: Select all

 
class ConfigReaderXML.... extends ConfigBase  {
   protected function _load(..);
} 
 
class ConfigBase  {
   public function load( ...) {
      set_error_handler(array($this, 'handleError');
      $this->_load( ...) //child method
      restore_error_handler();
   }
}
Now this obviously isn't ideal since our base class should not really ever call the child class. So how can we implement something like this and still suppress any output using composition? Again, hopefully this isn't too out of context (if not I'll remove it)
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: TDD workshop. Anybody welcome; Data storage abstraction.

Post by Chris Corbyn »

Why shouldn't the implementing classes handle that? How does the base class know what the child class is doing, what errors it will throw? Why would it care? :) You can still call set_error_handler() from within the enclosing class.

It may be the case that at some point down the line we do see common functionality and extract a superclass as part of a refactoring. But that's something that comes from refactoring which we'll get to later. We just want the simplest workable solution possible, and right now we don't seem to need a base class :)
Post Reply