28 October 2010

I Have Written Bug-Free Code Without TDD

There's an excellent blog post by Felix Geisendörfer which gives his experience report on using TDD. Kent Beck tweeted about it, so it's hopefully "going viral" right now. Read it here.

Reading his post, and some of the comments afterwards (when will I learn?!), got me to thinking about my own experiences with TDD, and I decided to share two of my own stories.

Once Upon a Time

I used to write bug-free software without TDD! (Look at me: I am so smart...)

My boss and I would play this game called "Who Wrote the Bug?" in which he would claim that a bug was in my code, and I would come back (after painstakingly debugging the whole product) and point out where it was in his code. (Hmmm...who was smarter?)

Humility Sets In

The software I was writing was a protocol stack that needed to convert between 8-bit and 9-bit bytes, or 16-bit and 36-bit words. I used the command-line/printf approach (System.out.println for the Java readers; Console.WriteLine for C# readers) to test stuff. The debugger, COFF, was okay, but testing through a debugger is such a pain.

If you haven't figured it out from the clues I've dropped, this was over 20 years ago. In retrospect, this was pretty tame stuff. I could keep track of what was going on in my head.

One can assume that software projects these days are more complex. Perhaps too complex for one person to keep track of all the little moving parts. It's enough for the team to have a grasp of the overall concepts, goals, and metaphors.

I would argue that even on a team with only three or four developers, eventually they're going to bump into each other's changes. They need a suite of microtests in order to maintain quality code.

The test frameworks like JUnit, NUnit, and RSpec essentially give you a way to let the computer check the "printf" output (figuratively speaking) for you. You record your assumptions, rather than manually checking them each time, and you can let those assumptions grow upon each other. You record them separately, rather than interspersing the following infamous abomination:
#ifdef DEBUG
printf("foo=%s", foo);
#endif

Two Years of Investment

In 2002 I worked on a team of four J2EE developers on a life-critical (i.e., "You break it, someone may die") application. After two years, we had built some pretty sophisticated reports and heuristics that helped hospitals determine who was most likely to survive an organ transplant.

We had over 10,000 tests. They all ran in about 12 minutes.

Imagine: All aspects of a complex, life-critical, team-built application could be thoroughly checked in 12 minutes. We could (we did) bring in someone new, and essentially say "today we will make whatever changes we need, even mere hours before the next release, as long as we run all the tests and see a 100% pass-rate!"

Twelve minutes, and we knew we hadn't broken any of our careful, 8-(plus-)person-year investment.

Rise to the Occasion

If you're sitting alone in your basement writing a game to sell to a VC, you may not care if it's maintainable. By all means, please continue using only manual testing techniques to verify your app. I'm not being snarky: This may be the optimal win-win approach for you and the VC. (The team that eventually maintains your app will curse your name and throw darts at your picture, but do you care?)

On the other hand, if you're part of a team, writing mission-critical software for a large or complex domain, and it needs to get beyond version 1.0; then please: Make sure you can always test everything, and quickly. TDD is the best--the only--way the industry has invented so far for doing this.

Anything else would be unprofessional.