Page 3 of 3

Re: Testing against production data

Posted: Wed Nov 05, 2008 9:19 am
by Jenk
inghamn wrote:
Jenk wrote: Or you can just use a DAO library, or use intelligent string comparison.. split the string and trim each word, etc.. to repeat myself once again.
The code we're talking about...what we're trying to test...*is* the DAO library. We're talking about testing that the ActiveRecord object did in fact update or delete all the stuff in the database that it was supposed to. Testing your DAO with simple string comparison is not going to find any problems.
The ActiveRecord is not the DAO. The DAO is the layer between your code/objects and the database, which is used to populate the AR's values. E.g. PDO. I.e. that instance variable called $this->db or whatever within your AR.

In an ideal world, even when testing the DAO, all you'd do is mock the stream and expect the predefined string/byte array.

Re: Testing against production data

Posted: Wed Nov 05, 2008 10:59 am
by allspiritseve
Jenk wrote:TDD is not new. It is 30+ years old.

The mantra of "Everything must be controlled within a test case, and have no external dependencies" is being broken.
I honestly don't know where you're getting your information, but I don't know enough about the history of programming to prove you wrong. Even if it has existed for "30+ years" (which I don't believe) there is still not a rigid structure or "mantra" that defines the RIGHT thing to do. Even if you look at best practices, people say to control what you can, and the things you can't control, move those tests to the boundary of your application. If you're testing business logic, sure, that's central to your application and should be controlled in a test case, but anything that interacts with a db is on the boundary, and thus, out of your control. You can't apply an "everything must be controlled mantra" to it, which means you have to adapt your testing practices. That, or leave your application exposed to errors because your test coverage isn't complete enough.

Re: Testing against production data

Posted: Wed Nov 05, 2008 11:04 am
by allspiritseve
Jenk wrote:The ActiveRecord is not the DAO. The DAO is the layer between your code/objects and the database, which is used to populate the AR's values. E.g. PDO. I.e. that instance variable called $this->db or whatever within your AR.

In an ideal world, even when testing the DAO, all you'd do is mock the stream and expect the predefined string/byte array.
Do you even know what an ActiveRecord is? Or PDO, for that matter? An ActiveRecord queries the database. PDO is just a wrapper for a database conection, it is not a DAO. If you have separate objects that query the database, outside of your domain objects, then you don't have an ActiveRecord. Typically you'll have a table data gateway or a data mapper.

Re: Testing against production data

Posted: Wed Nov 05, 2008 2:04 pm
by Jenk
DAO.. Data Access Object.. the Object used for Accessing Data..

All that makes an Active Record is it knows how to extract it's value from the data source and insert/update. It still has to communicate with a DAO to do this, otherwise you have more than just an AR..

Re: Testing against production data

Posted: Wed Nov 05, 2008 2:26 pm
by Jenk
allspiritseve wrote:
Jenk wrote:TDD is not new. It is 30+ years old.

The mantra of "Everything must be controlled within a test case, and have no external dependencies" is being broken.
I honestly don't know where you're getting your information, but I don't know enough about the history of programming to prove you wrong. Even if it has existed for "30+ years" (which I don't believe) there is still not a rigid structure or "mantra" that defines the RIGHT thing to do. Even if you look at best practices, people say to control what you can, and the things you can't control, move those tests to the boundary of your application. If you're testing business logic, sure, that's central to your application and should be controlled in a test case, but anything that interacts with a db is on the boundary, and thus, out of your control. You can't apply an "everything must be controlled mantra" to it, which means you have to adapt your testing practices. That, or leave your application exposed to errors because your test coverage isn't complete enough.
Kent Beck, the creator of sUnit (and jUnit with the help of Erich Gamma) said so in one of his seminars. I can't find a video for it, but it's out there somewhere.

This whole "application boundary" stuff is just a guise for "I don't want to address the real problem" in my eyes.

The real, absolute problem shown with unit testing anything with SQL strings has simply come down to one thing - the layout (or whatever you want to call it) of the string.

To be even more exact, It's brittle only because people are doing a direct equals comparison, and trying to assert on this, rather than asserting only that it would be accepted by the DB.

An SQL string assertion does not need to be equal, you just need to assert it will work. What I posted earlier address the problem linked and discussed.

Re: Testing against production data

Posted: Wed Nov 05, 2008 3:22 pm
by allspiritseve
Jenk wrote:It's brittle only because people are doing a direct equals comparison, and trying to assert on this, rather than asserting only that it would be accepted by the DB. An SQL string assertion does not need to be equal, you just need to assert it will work.
Just because an SQL statement is valid and would be accepted by the DB doesn't mean it will work as expected. What if you expect 1 row, and get 0? What if the table you want to use doesn't exist in the database? Any SQL statement is making assumptions about the database engine it is written for, and the schema it works with. In other words, your mock object can't just be an SQL debugger, it has to mock the entire engine. Which could be done, theoretically, but who wants to write that much code just to pass a test, when you have a database engine sitting right there that you can use?

Re: Testing against production data

Posted: Thu Nov 06, 2008 5:44 am
by Jenk
because that is for integration testing, not unit test - which is something that makes much more of a difference than you'd expect. My small snippet is smaller than what would be needed in setUp()/tearDown() if you use a db table anyway.. so that point is moot.

Re: Testing against production data

Posted: Thu Nov 06, 2008 4:37 pm
by josh
Given any SQL, someone could think of a different SQL implementation that achieves the same thing, ( returns the same results ) in a different way. Since the person who writes the test theoretically shouldn't be choosing any implementation details for the person writing code to pass the tests, including SQL in the test violates TFD as well as encapsulation. Test the SQL once and only once in the unit tests for the DAO. This ensures the correct SQL is being passed to the DBMS under fixed testable ranges of inputs. Then don't duplicate those tests when you go to test your other code. The tests for the DAO act as a harness for every other test. If the SQL fails only the DAO unit test will fail, where the problem actually is.

Testing the SQL does give you a stronger harness like mentioned, but I can't imagine where you would need your tests to be that responsive to implementation change. For instance going in and optimizing a query to run faster without changing its behavior, or re factoring a query.. should not cause tests to fail (change its outward appearance to the test). Tests are supposed to allow you to be able to refactor and the point of the test is to tell you whether you changed the behavior. IMO if you have a test going off every 2 seconds you couldn't trust your tests anymore. The test should see things as a black box.