Implementing Test-Driven Development with AI Agents

AI agents frequently struggle to write high-quality tests, often producing results that are vague, tautological, or overcomplicated. According to Jason Swett, this is largely because agents are trained on human-written examples that are often poor in quality. To solve this, Swett proposes using a structured "skill"—a set of guiding instructions—to force agents to follow a rational Test-Driven Development (TDD) process.

The Specify-Encode-Fulfill (SEF) Loop

Swett introduces the Specify-Encode-Fulfill (SEF) loop as a high-level alternative to the traditional red-green-refactor cycle. This framework ensures that the intent of the code is established before any implementation begins:

  1. Specify: Define the specifications for the desired functionality.
  2. Encode: Translate those specifications into automated, executable tests.
  3. Fulfill: Write the minimal amount of code necessary to satisfy the specifications.

Applying Canon TDD to AI Agents

To move from high-level intent to execution, Swett utilizes Kent Beck's Canon TDD. The process for the AI agent is as follows:

  • List Specifications: Create a comprehensive list of specifications within the current session's scope.
  • Incremental Encoding: Encode each item from the list as an automated test.
  • Minimal Implementation: Change the code only enough to make the current test failure disappear. This avoids "speculative coding," which risks introducing untested code.
  • Strict Separation: Refactor only after committing the behavior change. Behavior changes and refactoring must never be mixed.
  • Iteration: Repeat the process until the list of specifications is exhausted.

Enhancing Quality via Multi-Agent Review

Because a single agent may be biased toward its own implementation, Swett employs separate "review skills" to audit the output:

  • Test Design Review: A separate agent checks for violations of design principles, such as tests that focus on the means (implementation details) rather than the ends (outcomes).
  • Software Design Review: A general review to ensure the code adheres to broader software engineering principles, such as naming things accurately.

Swett also notes that Claude, in particular, responds well to analogies. By including an instruction to "clean the kitchen before making dinner" (meaning: refactor the existing codebase before adding new features), the agent frequently identifies necessary cleanup tasks that improve the overall design.

Community Perspectives on Agentic TDD

Discussion among developers reveals a significant divide regarding the utility of TDD in the age of LLMs:

Arguments for TDD

Some developers argue that testing is the most effective way to keep AI within guardrails. One user noted that spawning separate agents for review significantly increases code quality and decreases bugs, even if it increases token costs.

"Spawning separate agents to review the original agent's implementation results in a very noticeable increase in code quality and decrease in bugs."

Another developer found that the SEF loop is the only way to ensure that LLMs glue human-written specifications to code with confidence and maximum coverage.

Arguments Against TDD

Conversely, critics argue that TDD is "unnecessary bloat" or "snake oil" in the agentic era. Common objections include:

  • Token Cost: TDD cycles can balloon token usage and slow development velocity.
  • Hallucinations: Some users report that agents may create superficial tests that hallucinate success or trigger false positives, leading to unintentional refactoring.
  • Obsolescence: Some argue that modern models can "one-shot" implementations with high accuracy, making the iterative TDD process redundant.
  • Ownership: There is a concern that AI-authored tests create a sense of complacency, where humans no longer feel responsible for the test suite's correctness.

Validation Concerns

One critical point raised is the necessity of bug injection. A user pointed out that unless you validate that tests actually fail when bugs are introduced, you have only confirmed that the tests can pass, not that they are effective.

Sources