Page 1 of 1

Test for each function?

Posted: Sun Jan 21, 2007 1:06 pm
by alex.barylski
Edit2: How much should you test? Every execution path...how about making sure regex are doing their job cleaning data?

Edit: It just occured to me that functions are likely called in the order in which they are declared in the class (as it only makes sense) so as long as create is defined before return, this should work fine - as my test has just proven to me. Not sure if there is a better way to explicitly call a function before another?

I can see why it's not typical to test trivial getter/setter methods but I have a CRUD class in which it wouldn't make sense to test the functions independently (some of them anyways).

When I write a test...I'm looking to test for failure and every path of execution...my create functions are pretty simple in that they work or they don't - if they fail it's likely caused by DB connection failure nothing else!!!

So what I usually do is actually test the retreival functions, by first creating a record, returning it's content, checking that content against data supplied when created and then deleting that record and trying retrieval again and testing for failure. I also typically through in some update code.

so a single test (using my own testing methods not a framework) would typically look like:

1) Create a record using known values
2) Retreive that record
3) Check for proper values or existance
4) Update that record with new values
5) Retreive again and test values
6) Delete record
7) Attempt to retreive record

Each function on it's own it really trivial to test as it either works or it doesn't...DB connections or invalid schema are the only thing i'm testing for...

Now, using lastcraft unit test, I figure I *could* write tests for each CRUD operation - telling me which function has an invalid schema...or I could use my own approach and write something of a super test (as described above) which tests all methods in a consecutive manner...

Obviously that approach is messy and difficult to maintain and is why I've finally decided to take a kick at using a framework...

So I ask, is there a way using lastcraft unit test to instruct the framework to execute functions in a particular order? I need known data in the DB to test retreival operations and having the testCreate() called before testRetrieve() would allow me to accomplish this without having to have to reproduce create code inside the retreival functions - do I make sense?

Cheers :)

Re: Test for each function?

Posted: Sun Jan 21, 2007 2:48 pm
by feyd
Hockey wrote:Edit2: How much should you test? Every execution path...how about making sure regex are doing their job cleaning data?
All execution paths, both expected and unexpected arguments (if possible). Regex should be tested.
Hockey wrote:Edit: It just occured to me that functions are likely called in the order in which they are declared in the class (as it only makes sense) so as long as create is defined before return, this should work fine - as my test has just proven to me. Not sure if there is a better way to explicitly call a function before another?
Unless you're talking about set up and tear down functions, I don't recall being able to (unless you don't call them tests and simply call them from your actual tests.)
Hockey wrote:When I write a test...I'm looking to test for failure and every path of execution...my create functions are pretty simple in that they work or they don't - if they fail it's likely caused by DB connection failure nothing else!!!
That is something to test. The database calls themselves should probably be mocked.
Hockey wrote:Now, using lastcraft unit test, I figure I *could* write tests for each CRUD operation - telling me which function has an invalid schema...or I could use my own approach and write something of a super test (as described above) which tests all methods in a consecutive manner...
Tests should always be independent of one another. They may be called in any order.
Hockey wrote:So I ask, is there a way using lastcraft unit test to instruct the framework to execute functions in a particular order? I need known data in the DB to test retreival operations and having the testCreate() called before testRetrieve() would allow me to accomplish this without having to have to reproduce create code inside the retreival functions - do I make sense?
See above.

SimpleTest, not Lastcraft unit test. ;)

Re: Test for each function?

Posted: Sun Jan 21, 2007 3:51 pm
by alex.barylski
The database calls themselves should probably be mocked.
Hmmm...I've read a few articles on mocks and I'm still not sure I get the entire picture...thinking about this I was forced to ask myself...what is the purpose of a test (not according to TDD but my own reasoning for writing tests). In this case trivial SQL CRUD operations which don't return TRUE or FALSE but possibly just act as a subroutine, the purpose of a test is to give me immediate notice that something has gone wrong with one of two things.

1) the DB environment as in the connection arguments are wrong (which isn't important for testing I don't think(
2) Indicates immediately that while refactoring a schema I have missed a field somewhere.

The latter is what I'm really using tests for. How does a Mock - mock this functionality? How would a mock know whether my SQL statement was correct? I would have to write a SQL parser and have the schema somehow hardcoded into the mock object no???

It seems more logical to just use a temporary DB and create the tables on the fly and destroy them on object destruction (tearDown)???

The environment is always available to me and although maybe slower than a Mock might be...it's more accurate testing isn't it?
Tests should always be independent of one another. They may be called in any order.
I understand that, although it seems like it would produce duplicate code...I mean...in order to test an update, I need to create the record first...but if I'm already testing the create method (which creates a record) why would I re-create "another" record inside the update test. Same goes for deleting a record - which requires an existing record to delete.

So would I just not test create() explicitly and rely on delete and update to test create or just bite the bullet and write create code for each? it's not alot (single function call) but it's still duplicated somewhat.

Perhaps storing the PKID of the created record in a member variables of the test object and using that for update and delete?

p.s-I apologize for the SimpleTest/lastcraft unit test but I was in a rush when I wrote that and couldn't be bothered to check the actual name despite being right in front of me :P SimpleTest it is :)

Cheers :)

Posted: Sun Jan 21, 2007 5:10 pm
by feyd
Mocks only copy the interface of the functions. You tell the mock what to return for the call. This way you don't have to rely on another library or object working to test your code. The test groups should be fairly atomic in nature.
Hockey wrote:I understand that, although it seems like it would produce duplicate code...I mean...in order to test an update, I need to create the record first...but if I'm already testing the create method (which creates a record) why would I re-create "another" record inside the update test. Same goes for deleting a record - which requires an existing record to delete.
What if you, or someone else, decides to only test if the update paths are working? This is why each test sets up the struts to work within, then tears them down after it completes.

Posted: Mon Jan 22, 2007 4:39 am
by Maugrim_The_Reaper
Hmmm...I've read a few articles on mocks and I'm still not sure I get the entire picture...thinking about this I was forced to ask myself...what is the purpose of a test (not according to TDD but my own reasoning for writing tests). In this case trivial SQL CRUD operations which don't return TRUE or FALSE but possibly just act as a subroutine, the purpose of a test is to give me immediate notice that something has gone wrong with one of two things.
What's missing is that you don't note the core responsibility of the class. So it could be:

a) An abstraction class; one which is ultimately responsible for calling the PHP extension functions for a specific database
b) A Model class; modelling a specific row or table and delegating to a separate abstraction layer.

If testing a model, you should nearly always mock the abstraction layer it delegates to for access to the PHP database extensions. If an abstraction class, mocking the database will prove impossible (since the class requires a database in order to test it's functionality).

Should specify which one your posts refer to, and yes, a Model is an Abstraction layer if it directly uses a PHP extension. Though I'd venture this is a code smell since your class now has two responsibilities (handle Model, abstract Database extension) not just a single core one. This last option would explain most of your testing difficulties.
The latter is what I'm really using tests for. How does a Mock - mock this functionality? How would a mock know whether my SQL statement was correct? I would have to write a SQL parser and have the schema somehow hardcoded into the mock object no???
Unit tests are directed to a class's public expected behaviour. So, where is the SQL coming from? If the class is generating it, then the SQL is a class responsibility and you must test that for a specific set of input data, a specific and expected SQL statement is generated for each public method. Note, an expectation is not insurance for SQL validity - all the class cares about is that it's SQL is passed to the abstraction layer (or for an abstraction layer that it gets executed successfully).

If it's written by the programmer or is external data, then it's not possible to verify its validity (tests are only repeatable for FIXED inputs and outputs defined WITHIN the tests). The validity of SQL is the domain of either developer judgement (they are your tests) and the database (successful execution or not). Pinning test success on the output of an external database is for an abstraction layer - not a Model.
The environment is always available to me and although maybe slower than a Mock might be...it's more accurate testing isn't it?
Assuming this is a not an abstraction layer for a moment... Unit Tests document the expected behaviour of a class, and are therefore valid documentation for developers. If documentation is only useable on your personal platform and will fail on someone else's, then it's not very good documentation. If this is an abstraction layer, then you should make it clear (I would document inside the test class itself, as well as a README or Developer Documentation) what your platform is, and provide instructions on how to replicate it (versions, app names, specific config settings the tests are reliant on).

Notably, a Model class test is entirely accurate if its expected interaction with a mocked abstraction layer, as described by the unit tests, takes place. If you want more accuracy, and test both the Model and Abstraction classes in the same test then you introduce a dependency. The unit tests for one class, depend entirely on another class - you lose independent testing of each component and muddy the water. This is the reason Mocks exists - tests are like anything OOP, minimise dependencies.
I understand that, although it seems like it would produce duplicate code...I mean...in order to test an update, I need to create the record first...but if I'm already testing the create method (which creates a record) why would I re-create "another" record inside the update test. Same goes for deleting a record - which requires an existing record to delete.
Tests are classes. Classes have code smells. We solve code smells using Refactoring. If code is being duplicated in a test class, extract a new method. Just make certain it's completely isolated and shares no data/objects between calls.

Posted: Mon Jan 22, 2007 10:24 am
by alex.barylski
Maugrim_The_Reaper wrote:If testing a model, you should nearly always mock the abstraction layer it delegates to for access to the PHP database extensions. If an abstraction class, mocking the database will prove impossible (since the class requires a database in order to test it's functionality).
Hmmm...yes that could be causing the confusion... :P

The class is acting as a DAL which is later used by a model...it's offers an abstraction layer which provides a flex point for data access mediums (realistically only SQL will be used) although AdoDB is not needed or wanted.

Sounds like a mock wouldn't be appropriate here...but when writing tests for the model a mock of this abstraction layer would be? How would that work?

The DAL has an API like:

Code: Select all

class User{
  create(){}
  update(){}
  delete(){}
}
The model then later wraps the DAL...so the DAL would become the Mock? Mocking that API is rather trivial but I assume the Mock in this case would simply assume success when accessing the medium? For instance, when checking if a User exists (User::checkExists) would by default return TRUE?

Am I on the right track here with Mocks?

Posted: Mon Jan 22, 2007 11:45 am
by Maugrim_The_Reaper
I think you have an odd class structure ;), I'm pretty sure I'm not understanding it at least. An abstraction layer is usually unaware of database structure, and is solely concerned with enabling Models (or any ad-hoc code really) access the database through a single API (regardless of database/DBMS extension in use). PDO is popular for this, but even PDO needs some abstraction if you're really intent on supporting more than one database type (to handle the inevitable SQL differences). If you have abstraction at the level of database tables (in a Model), then the abstraction is going to be scattered across multiple classes (each Model needs it's own separate abstraction). User, being a specific table, is part of the Model.

I suppose the first question is how User gets from update() to a low level function like mysqli_query()? Is User delegating to another class, or are there multiple subclasses of User for each database option (or does your design suggest there would be in the future if someone wanted pgsql)? If User is directly running the low level database extension calls, then it's taking on both Model and Abstraction roles - which will be seen in the number of expanding subclasses, all sharing duplicate logical code for each supported database extension.

Sort of a return to my view of Model - I typically have three class families. The first is an unified API for database access (non-specific to a Model) similar in functionality to ADOdb. I also have a Data Access class which operates generic CRUD, find(), and sits as a layer between the Model and the Abstraction classes. The Model itself comprise a Row class for each table which detail valid fields, tracks data value changes and row existence (easier update/create decisions) and is extended like a Data Mapper with further methods, e.g. findUsersFromCountry(), or similar.