How do handle hard to test features?

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Re: How do handle hard to test features?

Post by Jenk »

You don't manual test every test run.. you manual test once during development, and apply it in your test. You need to know what your end goal (query) is before you can begin to generate it.

Or.. you create the code to generate, then see what the output is and test it (manually), then use that query (copy and pasted) to run future tests against.

It really is not hard to change the data within a query. And honestly.. "complex queries" are not complex.
User avatar
Eran
DevNet Master
Posts: 3549
Joined: Fri Jan 18, 2008 12:36 am
Location: Israel, ME

Re: How do handle hard to test features?

Post by Eran »

you manual test once during development, and apply it in your test
That is correct, but it goes against what you said so far. You test it once manually during development, and then write a test that allows you to test it automatically with the test suite. The way you suggested (copy-pasting a generated query) creates a very coupled test that I would need to update every time I change the logic of composing the query.

Those are dynamically generated queries I'm talking about. Not just parameter substitutions, but entire sections of the query that are generated dynamically according to some conditions. The most common scenario are different sortings that depend on aggregate functions / subqueries. Depending on the sort criteria, I might want to extract those expensive calculations to a different query to just add data if it's not needed for sorting.
And honestly.. "complex queries" are not complex.
This might sound very simple to you and you can test it manually very easily every time there's a change, but for me I'd still need to rely on unit-testing to assure everything is working as intended.
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: How do handle hard to test features?

Post by josh »

I'd throw that methodology out on the fact that if I decide to change database vendors all my tests would break.

I use that methodology when testing things that generate HTML sometimes though. If for some reason I did want to assert on my SQL test last is how I would do it. The way I do it instead though I am able to test first, which is another reason I favor doing it the way I do. I test thru the API of my persistence framework.

I think its important to have a pinch point then it is to have defect localization, at first. Once you have coverage, then by all means pin your tests down and assert on the actual SQL if you have time. But its just as easy to put a bug into the test as it is to have the bug occur in production that way!!

I just like to look at my tests as documentation. If I'm simply showing them sql I'm not providing any value other then duplicating the production code. On the other hand if I show them how to use the active record/data mapper/whatever I'm using to persist, that adds some documentation value, a "spec" if you will.
Post Reply