A sufficiently detailed spec is code

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail

That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolation/extrapolation machines. And their popularity is precisely because they are usually very good at filling in details: LLMs use their vast knowledge to guess what detail to generate, so the result usually makes sense.

This doesn't detract much from the main point of the article though. Sometimes the interpolated detail is wrong (and indeterministic), so, if reliable result is to be achieved, important details have to be constrained, and for that they have to be specified. And whereas we have decades of tools and culture for coding, we largely don't have that for extremely detailed specs (except maybe at NASA or similar places). We could figure it out in the future, but we haven't yet.

> A sufficiently detailed spec is code

This is exactly the argument in Brooks' No Silver Bullet. I still believe that it holds. However, my observation is that many people don't really need that level of details. When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec.

The vibe coding maximalist position can be stated in information theory terms: That there exists a decoder that can decode the space of useful programs from a much smaller prompt space.

The compression ratio is the vibe coding gain.

I think that way of phrasing it makes it easier to think about boundaries of vibe coding.

"A class that represents (A) concept, using the (B) data structure and (C) algorithms for methods (D), in programming language (E)."

That's decodeable, at least to a narrow enough distribution.

"A commercially successful team communication app built around the concept of channels, like in IRC."

Without already knowing Slack, that's not decodable.

Thinking about what is missing is very helpful. Obviously, the business strategic positioning, non technical stakeholder inputs, UX design.

But I think it goes beyond that: In sufficiently complex apps, even purely technical "software engineering" decisions are to some degree learnt from experiment.

This also makes it more clear how to use AI coding effectively:

* Prompt in increments of components that can be encoded in a short prompt.

* If possible, add pre-existing information to the prompt (documentation, prior attempts at implementation).

I think it's only a matter of time before people start trying to optimize model performance and token usage by creating their own more technical dialect of English (LLMSpeak or something). It will reduce both ambiguity and token usage by using a highly compressed vocabulary, where very precise concepts are packed into single words (monads are just monoids in the category of endofunctors, what's the problem?). Grammatically, expect things like the Oxford comma to emerge that reduce ambiguity and rounds of back-and-forth clarification with the agent.

The uninitiated can continue trying to clumsily refer to the same concepts, but with 100x the tokens, as they lack the same level of precision in their prompting. Anyone wanting to maximize their LLM productivity will start speaking in this unambiguous, highly information-dense dialect that optimizes their token usage and LLM spend...

A spec is an envelope that contains all programs that comply. Creating this spec is often going to be harder than writing a single compliant program.

Since every invocation of an LLM may create a different program, just like people, we will see that the spec will leave much room for good and bad implementations, and highlight the imprecision in the spec.

Once we start using a particular implementation it often becomes the spec for subsequent versions, because it's interfaces expose surface texture that other programs and people will begin to rely on.

I'm not sure how well LLMs will fare are brownfield software development. There is no longer a clean specification. Regenerating the code from scratch isn't acceptable. You need TPS reports.

In my experience with “agentic engineering” the spec docs are often longer than the code itself.

Natural language is imperfect, code is exact.

The goal of specs is largely to maintain desired functionality over many iterations, something that pure code handles poorly.

I’ve tried inline comments, tests, etc. but what works best is waterfall-style design docs that act as a second source of truth to the running code.

Using this approach, I’ve been able to seamlessly iterate on “fully vibecoded” projects, refactor existing codebases, transform repositories from one language to another, etc.

Obviously ymmv, but it feels like we’re back in the 70s-80s in terms of dev flow.

Trying to go the Spec -> LLM route is just a lost cause. And seems wasteful to me even if it worked.

LLM -> Spec is easier, especially with good tools that can communicate why the spec fails to validate/compile back to the LLM. Better languages that can codify things like what can actually be called at a certain part of the codebase, or describe highly detailed constraints on the data model, are just going to win out long term because models don't get tired trying to figure this stuff out and put the lego bricks in the right place to make the code work, and developers don't have to worry about UB or nasty bugs sneaking in at the edges.

With a good 'compilable spec' and documentation in/around it, the next LLM run can have an easier time figuring out what is going on.

Trying to create 'validated english' is just injecting a ton of complexity away from the area you are trying to get actual work done: the code that actually runs and does stuff.

The cognitive dissonance comes from the tension between the-spec-as-management-artifact vs the-spec-as-engineering-artifact. Author is right that advocates are selling the first but second is the only one which works.

For a manager, the spec exists in order to create a delgation ticket, something you assign to someone and done. But for a builder, it exists as a thinking tool that evolves with the code to sharpen the understanding/thinking.

I also think, that some builders are being fooled into thinking like managers because ease, but they figure it out pretty quickly.

Natural language is fluid and ambiguous while code is rigid and deterministic. Spec-driven development appears to be the best of both worlds. But really, it is the worst of both. LLMs are language models - their breakthrough capability is handling natural language. Code is meant to be unambiguous and deterministic. A spec is neither fluid nor deterministic.

I agree with the overall structure of the argument but I like to think of specifications like polynomial equations defining some set of zeroes. Specifications are not really code but a good specification will cut out a definable subset of expected behaviors that can then be further refined with an executable implementation. For example, if a specification calls for a lock-free queue then there are any number of potential implementations w/ different trade-offs that I would not expect to be in the specification.

I don't agree. The code is much more than the spec. In fact, the typical project code is 90% scaffolding and infrastructure code to put together and in fact contains implementation details specific to the framework you use. And only 10% or less is actual "business logic". The spec doesn't have to deal with language, framework details, so by definition spec is the minimum amount of text necessary to express the business logic and behaviour of the system.

> On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

I guess many of us quality for british parliament.

I recently left this comment on another thread. At the time I was focused on planning mode, but it applies here.

Plan mode is a trap. It makes you feel like you're actually engineering a solution. Like you're making measured choices about implementation details. You're not, your just vibe coding with extra steps. I come from an electrical engineering background originally, and I've worked in aerospace most of my career. Most software devs don't know what planning is. The mechanical, electrical, and aerospace engineering teams plan for literal years. Countless reviews and re-reviews, trade studies, down selects, requirement derivations, MBSE diagrams, and God knows what else before anything that will end up in the final product is built. It's meticulous, detailed, time consuming work, and bloody expensive.

That's the world software engineering has been trying to leave behind for at least two decades, and now with LLMs people think they can move back to it with a weekend of "planning", answering a handful of questions, and a task list.

Even if LLMs could actually execute on a spec to the degree people claim (they can't), it would take as long to properly define as it would to just write it with AI assistance in the first place.

Safety-critical perspective:

Specification means requirements. I like EARS [0] syntax for requirements.

e.g. "while an error is present, the software shall ignore keypresses"

Requirements are not code at all, they are expectations about what the code does; it is the contract about what developers will be held accountable to. Putting implementation details in requirements is a rookie mistake because it takes agency away from the engineers in finding the best solution.

The spec discussed in this article is more akin to the level of detail appropriate in an interface control document (ICD). Very common for a requirement to declare the software shall be compliant to a revision of an ICD.

My own thoughts are: like a good systems engineer that recognizes the software engineers know their domain better than they, we should write specification for AI that leaves room for it to be more clever than we ourselves are. What's the point of exhaustive pseudocoding, it's worse coding. Align on general project preferences, set expectations, and concentrate effort on verifying.

[0]: https://alistairmavin.com/ears/

I've been trying codex and claude code for the past month or so. Here's the workflow that I've ended up with for making significant changes.

- Define the data structures in the code yourself. Add comments on what each struct/enum/field does.

- Write the definitions of any classes/traits/functions/interfaces that you will add or change. Either leave the implementations empty or write them yourself if they end up being small or important enough to write by hand (or with AI/IDE autocompletion).

- Write the signatures of the tests with a comment on what it's verifying. Ideally you would write the tests yourself, specially if they are short, but you can leave them empty.

- Then at this point you involve the agent and tell it to plan how to complete the changes without barely having to specify anything in the prompt. Then execute the plan and ask the agent to iterate until all tests and lints are green.

- Go through the agent's changes and perform clean up. Usually it's just nitpicks and changes to conform to my specific style.

If the change is small enough, I find that I can complete this with just copilot in about the same amount of time it would take to write an ambiguous prompt. If the change is bigger, I can either have the agent do it all or do the fun stuff myself and task the agent with finishing the boring stuff.

So I would agree with the title and the gist of the post but for different reasons.

Example of a large change using that strategy: https://github.com/trane-project/trane/commit/d5d95cfd331c30...

No, a spec is not code. It's possible to describe simple behavior that's nevertheless difficult to implement. Consider, say,

  fn sin(x: f16) -> f16

There are only 64k different f16s. Easy enough to test them all. A given sin() is either correct or it's not.

Yet sin() here can have a large number of different implementations. The spec alone under-determines the actual code.

It helps to decouple the business requirements from the technical ones. It's often not possible to completely separate these areas, but I've been on countless calls where the extra technical detail completely drowns out the central value proposition or customer concern. The specification should say who, what, where, when, why. The code should say how.

The code will always be an imperfect projection of the specification, and that is a feature. It must be decoupled to some extent or everything would become incredibly brittle. You do not need your business analysts worrying about which SQLite provider is to be used in the final shipped product. Forcing code to be isomorphic with spec means everyone needs to know everything all the time. It can work in small tech startups, but it doesn't work anywhere else.

I agree with this so much. And on top of this, I have the strong feeling that LLMs are BETTER at code than they are at english, so not only are you going from a lossy formate to a less-leossy format, you are specifying in a lossy, unskilled format.

For this to be true, we should be able to

- Delete code and start all over with the spec. I don't think anyone's ready to do that.

- Buy a software product / business and be content with just getting markdown files in a folder.

>> Misconception 1: specification documents are simpler than the corresponding code

I used to be on that side of the argument - clearly code is more precise so it MUST be simpler than wrangling with the uncertainty of prose. But precision isn't the only factor in play.

The argument here is that essential complexity lives on and you can only convert between expressions of it - that is certainly true but it's is overlooking both accidental complexity and germane complexity.

Specs in prose give you an opportunity to simplify by right-sizing germane complexity in a way that code can't.

You might say "well i could create a library or a framework and teach everyone how to use it" and so when we're implementing the code to address the essential complexity, we benefit from the germane complexity of the library. True, but now consider the infinite abstraction possible in prose. Which has more power to simplify by replacing essential complexity with germane complexity?

Build me a minecraft clone - there's almost zero precision here, if it weren't for the fact that word minecraft is incredibly load bearing in this sentence, then you'd have no chance of building the right thing. One sentence. Contrast with the code you'd have to write and read to express the same.

Why is everyone still talking about markdown files as the only form of spec? The argument is true for text-based specs, but that's not the only option. Stop being so text-file-brained?

This article is really attacking vague prose that pushes ambiguity onto the agent - okay, fair enough. But that's a tooling problem. What if you could express structure and relationships at a higher level than text, or map domain concepts directly to library components? People are already working on new workflows and tools to do just that!

Also, dismissing the idea that "some day we'll be able to just write the specs and the program will write itself" is especially perplexing. We're already doing it, aren't we? Yes, it has major issues but you can't deny that AI agents are enabling literally that. Those issues will get fixed.

The historical parallel matters here as well. Grady Booch (co-creator of UML) argues we're in the third golden age of software engineering:

- 1940s: abstracted away the machine -> structured programming

- 1970s: abstracted away the algorithm -> OOP, standard libraries, UML

- Now: abstracting away the code itself

Recent interview here: https://www.youtube.com/watch?v=OfMAtaocvJw

Each previous transition had engineers raising the same objections: "this isn't safe", "you're abstracting away my craft". They were right that something was lost, but wrong that it was fatal. Eventually the new tools worked well enough to be used in production.

There's essential complexity and accidental complexity.

A sufficiently detailed spec need only concern itself with essential complexity.

Applications are chock-full of accidental complexity.

There are two kid of specs, formal spec, and "Product requirements / technical designs"

Technical design docs are higher level than code, they are impricise but highlight an architectural direction. Blanks need to be filled in. AI Shines here.

Formal specs == code Some language shine in being very close to a formal spec. Yes functional languages.

But lets first discuss which kind of spec we talk about.

This won't age well, or my comment won't age well. We'll see!

> Misconception 1: specification documents are simpler than the corresponding code

That is simply not true. There is a ton of litterature around inherent vs accidental complexity, which in an ideal world should map directly to spec vs code. There are a lot of technicalities in writing code that a spec writer shouldn't know about.

Code has to deal with the fact that data is laid out a certain way in ram and on disk, and accessing it efficiently requires careful implementation.

Code has to deal with exceptions that arise when the messiness of the real world collides with the ideality of code.

It half surprises me that this article comes from a haskell developer. Haskell developers (and more generally people coming from maths) have this ideal view of code that you just need to describe relationships properly, and things will flow from there.

This works fine up to a certain scale, where efficiency becomes a problem.

And yes, it's highly probable that AI is going to be able to deal with all the accidental complexity. That's how I use it anyways.

I've heard various suggestions of only committing spec.md or change requests in the git repo and using that as source of truth.

We have spent decades working on reproducible builds or deterministic compilation. To achieve this, all steps must be deterministic. LLMs are not deterministic. You need to commit source code.

This is relatable.

I did a side project with a non-technical co-founder a year ago and every time he told me what he wanted, I made a list of like 9 or 10 logical contradictions in his requirements and I had to walk him through what he said with drawings of the UI so that he would understand. Some stuff he wanted me to do sounded good in his head but once you walk through the implementation details, the solution is extremely confusing for the user or it's downright physically impossible to do based on cost or computational resource constraints.

Sure, most people who launched a successful product basically stumbled onto the perfect idea by chance on the first attempt... But what about the 99% others who fell flat on their face! You are the 99% and so if you want to succeed by actual merit, instead of becoming a statistic, you have to think about all this stuff ahead of time. You have to simulate the product and business in detail in your mind and ask yourself honestly; is this realistic? Before you even draw your first wireframe. If you find anything wrong with it, anything wrong at all; it means the idea sucks.

It's like; this feature is too computationally and/or financially expensive to offer for free and not useful enough to warrant demanding payment from users... You shouldn't even waste your time with implementation; it's not going to work! The fundamental economics of the software which exists in your imagination aren't going to magically resolve themselves after implementing in reality.

Translating an idea to reality never resolves any known problems; it only adds more problems!

The fact is that most non-technical people only have a very vague idea of what they want. They operate in a kind of wishy washy, hand-wavy emotion-centric environment and they think they know what they're doing but they often don't.

This is laid out in “the code is the design” - https://www.developerdotstar.com/mag/articles/reeves_design_... by jack reeves.

Like they say “everything comes round again”

I agree with most of what the author is saying, but the slogan that "a sufficiently detailed spec is code" can be misunderstood as "a sufficiently detailed spec is a program". The statement is only true if you read "code" as "statement in a formal language". Here's a (sketch of a) specification for a compiler:

> For every specification satisfied by the input program, the output program satisfies the same specification.

This is not a program and it does not become a program once you fill in the holes. Making the statement precise clearly requires a formal language, but that language can work at a higher level of abstraction than a programming language. So yes, a specification can absolutely be simpler than a program that implements it.

A sufficiently detailed spec was actually a small step in the path to functional code.

Then came all sorts of shenanigans, from memory management to syntax hell, which took forever to learn effectively.

This stage was a major barrier to entry, and it's now gone — so yeah, things have indeed changed completely.

> Misconception 1: specification documents are simpler than the corresponding code

Anyone who studied software engineering, should know that specification doesn’t bother with implementation details of the underlying technology.

Things such as quite specific engine are used, are the contents of an encapsulated subsystem.

Proper software engineering specification is incompatible with a hacker culture and picking technology beforehand is a bad practice. It’s much closer to waterfall than to C4.

However, the last 20 years we got software building blocks which impose system architectural restrictions: frameworks. And also pieces of software which are half cooked systems.

Far are the days of requirements, preconditions, postconditions and invariants, network diagrams and entity relationship models.

I tried myself to make a language over an agent's prompt. This programing language is interpreted in real time, and parts of it are deterministic and parts are processed by an LLM. It's possible, but I think that it's hard to code anything in such a language. This is because when we think of code we make associations that the LLM doesn't make and we handle data that the LLM might ignore entirely. Worse, the LLM understands certain words differently than us and the LLM has limited expressions because of it's limits in true reasoning (LLMs can only express a limited number of ideas, thus a limited number of correct outputs).

It seems like everyone has a very different idea of what spec means in agentic coding.

To me, spec answers the what, the plan answers the how, and in what order, build packets answer the how but with more granularity.

In most cases, you should only care about the what. How it gets done (plan) is simply an implementation detail that you should not care about the same way automated tests should not care about them.

What you prescribe in spec is that data must pass from A to B through C, preserved in D and presented in E in shape of F. It's much easier to write (and change) this in spec than in say, Rust.

A corollary of this statement is that code without a spec is not code. No /s, I think that is true - code without a spec certainly does something, but it is, by the absence of a detailed spec, undefined behavior.

I am developing my own programming language, but I have no specification written for it. When people tell me that I need a specification, I reply that I already have one - the source code of the language compiler.

I agree to this, with the caveat that a standard is not a spec. E.g.: The C or C++ standards, they're somewhat detailed, but even if they were to be a lot more detailed, becoming 'code' would defeat the purpose (if 'code' means a deterministic turing machine?), because it won't allow for logic that is dependent on the implementer ("implementation defined behavior" and "undefined behavior" in C parlance). whereas a specification's whole point is to enforce conformance of implementations to specific parameters.

Maybe an argument can be made that this definitely holds for some areas of the feature one is building. But in ever task there might be areas where the spec, even less descriptive than code, is enough, because many solutions are just „good enough“? One example for me are integration tests in our production application. I can spec them with single lines, way less dense than code, and the llms code is good enough. It may decide to assert one way or another, but I do not care as long as the essence is there.

Could be that the truth is somewhere in between?

Such amazing writing. And clear articulation of what I’ve been struggling to put into words - almost having to endure a mental mute state. I keep thinking it’s obvious, but it’s not, and this article explains it very elegantly.

I also enjoyed the writing style so much that I felt bad for myself for not getting to read this kind of writing enough. We are drowning in slop. We all deserve better!

IMHO, LLMs are better at Python and SQL than Haskell because Python and SQL syntax mirrors more aspects of human language. Whereas Haskell syntax reads more like a math equation. These are Large _Language_ Models so naturally intelligence learned from non-code sources transfers better to more human like programming languages. Math equations assume the reader has context not included in the written down part for what the symbols mean.