- Testing
- AI
Birgitta Böckeler recently ran an experiment, published on martinfowler.com: do AI coding agents produce better work when you force them through a strict Test-Driven Development loop? The answer was uncomfortable. She found "no clearly discernable difference" between the TDD and non-TDD runs. Non-TDD solutions sometimes scored a little higher on design and test quality. The TDD runs cost roughly three to eight and a half times more tokens to get there.
Worth stating plainly, because she does: this is a small sample, and she is explicit that it is "obviously a very small sample size, so take it with a grain of salt." It is a signal, not a settled result.
At base32, it matches what we see. So here is our take: separate the ritual from the artifact.
TDD was always two things at once. It is a process — red, green, refactor — and it is a product, a suite of tests that pin behaviour down. Humans need the process. The small steps manage fear. The up-front test forces you to decide what "done" means before you write the code. An agent feels none of that. It has read millions of finished functions and very few worked TDD transcripts, so the loop is not its natural shape. To make it perform the loop is theatre.
The product, though, is more valuable than ever.
When an agent writes the implementation, you can no longer trust the process behind it, because there is no human judgement inside each step. The tests become the contract you check the output against. They are the thing you actually read. So we still write tests first — but as a specification the agent must satisfy, not as a cadence it must mime. We fix the expected behaviour, hand it over, and let the model reach green however it likes.
Böckeler's practical suggestions point the same way: mutation testing to watch regression quality, and frozen scenarios that demand re-approval when they break. These judge outcomes, not method.
The headline is not "TDD is dead." It is narrower and more useful. The discipline of stating expected behaviour up front survives. The step-by-step ceremony around it does not.
Written by base32.