alt Hacker NewsThis is a decent approach. My concern with TDD is that writing tests necessarily implies designing an API upon which those tests operate. Here, the agent is instructed to "not write code, write tests", and yet, in doing so it defines an API. This will cause the AI to hallucinate the API. Layering in yet more tests on top of this will cause that API to deform in strange ways that pass tests but that the adversary will not be able to cope with because it runs too late in the VSDD process.
I've seen this exact process play out in my own work. The AI generates code and tests that pass with high code coverage and honors invariants set by spec. I look at the code and find a rats nest / ball of mud that will cost 10x more tokens to enhance should I ever need to add a feature.
So, I think you're on to something, but I think the process might be discounting extensibility and resilience under change.
Replies
I see what you're saying. Modularity and interfaces are really important between the different aspects of what's being developed. And it is worth putting time into the question "if another will use this, what would they potentially use it for, and why?". It doesn't mean that it needs to be built now - but considering that and ensuring that the planned code executes against that is a good strategy.
> My concern with TDD is that writing tests necessarily implies designing an API upon which those tests operate
It really forces you to do outside-in testing; I usually describe the kind of API I want when chatting with the agents. For example, the CLI options, the routes that might be useful for an API, etc.
> I look at the code and find a rats nest / ball of mud that will cost 10x more tokens to enhance should I ever need to add a feature.
Agreed, I don't know if there are good forcing functions to avoid complexity. The providers have a huge incentive to have you waste your tokens (for example when it re-outputs the complete file when you ask for a tiny change).