11 October 2007 - 17:41XP Testing

Yet another short post on a topic dear to XP-developers everywhere.

Testing as documentation.
Testing is not documentation. Or rather put is poor documentation. Sure you encapsulate the desired behavior of your application in tests, but this doesn’t mean that you actually document it. One of the reasons that testing is not documentation because in order to get the desired behavior out of a test (especially a non-commented test) you need to reverse engineer the test. As you can imagine this is a very error-prone process. You could make your tests more readable by commenting them but those comments are the actual documentation and not the tests themselves. If tests were documentation then documenting the tests would be as documenting the documentation, a mind-numbing concept on its own.

Tests, and unit tests especially, are low-level entities, they are close to the code. Documentation requires views of the application at different levels, from the low-level (small pieces of code) to high-level (functionality).While you could put together a series of tests which would test the application at a higher-level (and you should probably do so in order to test your application at this level) I don’t think you can navigate the tests easily in order to determine which are high-level, which are low-level, etc…

Yet another reason for which testing is not documentation is the fact that documentation and testing have 2 different reasons to exist: documentation is about informing a developer about the application (and this includes a lot more than functionality) while testing is about making sure that certain desired functionality is being implemented correctly. Documenting an application requires that you express some concepts in the most efficient and less error-prone way so that a developer can pick up the documentation and understand it as easily as possible.
This process of encapsulating knowledge in order to make it available to someone is a very tricky process and the failure of various methodologies would be witness to this. My impression is that this is a process which requires a lot of research, particularly in knowledge management, and that the test as documentation solution advanced by the XP-guys is just another milestone on the road to greater confusion.
Some people may find test as documentation to work for them, but I think they are a very small minority. Documenting an application is a process that is particular to a development team and assuming that all development teams can be reduced to a common denominator (such as test-is-documentation) is a pretty big mistake.

Test-driven development.
Yet another interesting use of testing. First of all, do you really want to start new development with a compilation error (talk about starting on the right foot)? Probably not.
The biggest problem that I have with TDD is that it encourages you to write a test before you write a class this test is supposed to test. As you expect, you get a compilation error. However, one problem that arises is that you reference the target class in the test itself, and you do this by usually putting the target class in a dummy package. Once all is fine you re-factor your code base and the target class gets assigned to its proper package.
This is a classic example of designing after coding: moving the target class to its proper package is a design issue (assigning a package to a class is basically managing the relationships between your classes) which you get to do after the class is coded. I think that this is a pretty bad arrangement, ideally you design before you code.
I know that design is seriously looked down in the XP community and TDD is symptomatic of this attitude.