Understanding Spec-Driven Development and Its Missing Testing Layer
The Spec-Driven Development Workflow
In spec-driven development, the written specification is the foundation of the workflow. The spec is authored by the development team, and then an AI agent generates the implementation from it. When requirements change, the spec is updated, and the code is regenerated, rather than editing the code directly. This approach provides a clear and persistent record of intent, which is essential for maintaining the integrity of the software.
Open-Source Spec-Driven Development Tools
While commercial tools are well-known, there are also open-source options available. GitHub Spec Kit is a popular open-source tool, but there are other interesting tools that are often overlooked. For example, OpenSpec, BMAD-METHOD, Spec Kitty, and Specs CLI are all open-source tools that support spec-driven development. Each of these tools has its own unique features and approaches to structuring the spec and driving the AI agent.
OpenSpec
OpenSpec is a lean tool that allows developers to write delta specs, which describe only the changes made to the software. This approach makes it a strong fit for brownfield work on existing codebases. The strict three-phase flow of propose, apply, and archive makes it easy to manage changes and maintain a growing source-of-truth document.
BMAD-METHOD
BMAD-METHOD is a heavyweight tool that orchestrates a team of specialized AI agents across the full lifecycle of the software development process. It is suitable for complex greenfield projects where structure and organization are essential. The tool provides a comprehensive framework for managing the development process, from specification to deployment.
Spec Kitty
Spec Kitty is a repo-native CLI that runs the whole loop of spec to plan to tasks to review to accept to merge. It includes a Kanban dashboard and git worktrees built in, making it a powerful tool for managing the development process. Spec Kitty drives multiple AI agents, including Claude Code, Cursor, Gemini, Codex, and Copilot, providing flexibility and choice.
Specs CLI
Specs CLI is an agentic command-line tool built around traceability. It turns an idea into a spec, a plan, and tasks while keeping a link from each piece of code back to the requirement it came from. This tool provides a clear and transparent way to manage the development process, ensuring that every piece of code is linked to a specific requirement.
The Limitations of Vibe Coding
Vibe coding broke down as projects grew because it lacked a durable record of intent. The prompt was thrown away, and the code was the only artifact, making it difficult to determine whether a behavior was a feature or an accident. Spec-driven development addresses this limitation by providing a clear and persistent record of intent.
The Blind Spot in Spec-Driven Development
The spec is a set of claims about how the software should behave, and the AI agent generates code that it believes satisfies those claims. However, most teams fail to verify that the generated code actually meets the specifications. This blind spot can lead to errors and defects in the software.
Testing Spec-Driven Development Output
The fix for this blind spot is to execute the acceptance criteria in the spec against the running software. This approach ensures that the software meets the specifications and provides a clear and reliable way to verify the correctness of the code. As a pro-tip, using a tool like TestSwiftly can help automate this process, providing a comprehensive testing framework that can verify the correctness of the code and ensure that it meets the specifications.
Where Testing Fits in an SDD Workflow
The clean version of the workflow has four steps: write the spec, generate the code, generate behavioral coverage from the acceptance criteria, and verify the running software against it before merge. This approach ensures that the software meets the specifications and provides a clear and reliable way to verify the correctness of the code.
FAQs
What is Spec-Driven Development?
Spec-driven development is an AI coding workflow where a written specification is the source of truth. The spec is authored by the development team, and an AI agent generates the code from it.
How is Spec-Driven Development Different from Vibe Coding?
Vibe coding uses a loose prompt and keeps only the code, while spec-driven development uses a structured, persistent spec that can be reviewed and versioned before any code is generated.
Does Spec-Driven Development Remove the Need for QA?
No, spec-driven development does not remove the need for QA. The acceptance criteria in the spec still need to be verified against the live application, which is a testing job, not a generation job.
Who Verifies that AI-Generated Code Matches the Spec?
Nobody, by default, which is the blind spot. Closing it means turning the spec’s acceptance criteria into end-to-end tests and running them against the running product on every regeneration.
What Tools Support Spec-Driven Development?
The commonly cited tools are commercial, but there are also open-source options available, including OpenSpec, BMAD-METHOD, Spec Kitty, and Specs CLI.
Can the Same Spec be Used to Generate Tests?
Yes, and it should be. The acceptance criteria in a spec are already behavioral assertions in the form of given, when, then, which is the shape of an end-to-end test. Generating coverage from the same criteria that drove the code is what verifies that the two actually agree.
To further enhance the testing process, consider utilizing TestSwiftly for automated testing, ensuring that your software meets the required specifications and functions as intended.