Understanding is the new bottleneck

I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck.

It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program management is a job.

All those behaviors engineers hated about their bosses that kept them away from being focused on the code...they're starting to feel what it's like on the other side and reinventing the solutions instead of just reading a book about engineering management. Maybe we'll rebrand program management to "understanding ops" or something.

I wonder what AI would say about us if given the tokens to complain.

We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.

Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.

I agree with the problem but not the solutions.

The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.

Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.

(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)

LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.

"I read the code." -Mitchell Hashimoto

Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.

It's more like this:

We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely.

Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding.

We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less.

If the whole thing materializes in one day, that doesn't work; the approach is gone.

You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.

In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.

I am so dying to read more about the new/current/real bottleneck!

Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!

For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.

I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?

I found the linked article "books don't work" from the quiz part highly interesting

https://andymatuschak.org/books/

It explains a lot and works really well.

I tried out in ChatGPT with a simple prompt:

> ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns.

Really fun experience.

I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve.

I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.

While the tips are good to handle the volume, I still think this sets code owner on a dangerous path.

AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.

The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.

Low value comment, so I apologize, but it's funny to note how many bottleneck articles there are now as a result of AI adoption. Lots of new bottlenecks.

Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss.

If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.

How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are

Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).

"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side."

It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it.

Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.

Improving code understanding is the main focus of my work and thinking right now. If we want to make advances I believe that we should rely more heavily on one key quality of the program code: It is meant to be executed.

Here are some ideas:

1. Time travel debugging. Reading a PR just like a wall of text is difficult, but what if you could step through the PR and see the state at a given line for some test executions? Time travel debugging can make this possible. You would collect a debug trace and use it to overlay the PR diff with additional controls and information to resemble a debugger's UI. I was part of the team behind Codetracer (https://github.com/metacraft-labs/codetracer) who is trying to work in this direction.

2. Test suites and coverage. We don't use them enough for understanding right now. The test suite encodes what features the code is supposed to have, and the coverage tells us where in the code those features are implemented. I'm playing with an idea about this here: http://atlas.vihren.dev When we intersect coverages for the different test cases we can arrive at code segments which represent "atomic behaviors" present in the code. They form a mathematical structure which can be represented as a graph. I am currently exploring what value we can extract from it for the benefit of both humans and agents.

My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be.

So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.

A small bone to pick, but describing an AI that is operating autonomously as creative seems wrong - at best, this process is accretive, because the AI is adding and adding, but has no ability or incentive to shape its output toward something a human would find of value. Value is subjective (individual), changing over time. An AI doesn't know when it needs to be taking away - removal is a key part of the creative process.

There is a body of knowledge in unit testing, integration testing, static analysis, model checking, formal methods, fuzz testing, … (what other techniques for building assurance in our code have I forgotten). And LLMs can be put to use towards all of these methods. But sure, we just need to think harder to solve all our problems. Velocity of code goes up. Velocity of testing can also go up. It’s just not as fun or glamorous.

I think the ideas presented in the article are interesting, but at times the Notion references were a bit much. The author disclosed that they work there, fine, but at times it felt like it was more about showing off a Notion feature (e.g. embedded interactive HTML).

I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand.

The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. The SPEC file(s) should contain all the details and behavior you care about, and anything you don't care about is up to the agent to decide. If you don't like what the agent picked, _put it in the spec file_.

Critically, _you_ must write the SPEC file. You ensure understanding by doing so. You can of course ideate with the agent, but it's your ideas, in your words, specified by you. This also makes it a great source of documentation when you come back later and have to remember wtf is going on in this codebase.

> Why? Why understand?

Because (in the just-barely-possible universe where anyone is stupid enough to make me a manager) I will fire you if I ever find out you don't.

What possible good can come from letting people deploy things over which they do not, in any sense, have cognitive ownership?

"new"? The reason for all the old bottlenecks is that you had to understand as you went along.

Understanding has always been the bottleneck, everywhere for everything. And now, with this new realization, are you going to finally realize that Communications and your skills with it are basically everything?

Analysis and synthesis are two sides of human intelligence. Current form of generative artificial intelligence is very powerful in synthesizing information and producing artifacts, human will lose this battle.

Understanding is about analyzing and we still have some tools to help us, such as type checking, testing, etc. To some extent this can be automated but needs to be maintained automatically to match the flood of synthetic artifacts.

“Synthetic analysis” is an oxymoron and could lead to hallucinations and irrelevance.

Understanding has always been the bottleneck.

In a team: yups.

Me with my LLMs: still.

I guess it depends on the context and environment, right? In many corporates, the bottleneck for me has always been specs and testing. I have daily examples. I had to explain it to leadership like this: "There are far more ways things can go wrong than right".

> Aren't we supposed to be taking ourselves out of the loop now, and letting the agents loop themselves

That's what Boris Cherney said. Boris Cherney is not your dad. You don't have to listen to him.

Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did).

Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?

If you start with a spec you understand at the beginning then you don't need the LLM to generate high-level information about the changes at review time.

The grilling (grill-with-docs) skills [1] are amazing for ensuring you produce a through spec that covers all the edge cases. The /code-review skill from there helps ensure that the code changes meet the spec.

I use an intermediate detailed plan stage (done by a more expensive model) before implementation. Information from that plan is posted on the PR to give pretty much all the intermediate level context reviewers need.

I do like incorporating the idea of this article into my flow- that the spec and PR context could be presented in a more educational way.

[1] https://github.com/mattpocock/skills

Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.

Understanding was always the bottleneck. It’s just now the understanding is backloaded instead of frontloaded.

ie. understanding it AFTER it’s already generated rather than before you wrote the code by hand

Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code.

The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.

This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.

This talk is also available on YouTube: https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi

Claude make an entire app for me to describe this security contract change by pretending I’m in a Zelda game and only use funny metaphors because i’m bored and can’t read typescript

I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.

Is there a markdown version of the `/explain-diff` skill? The page says there are HTML, markdown, and Notion versions, but I just see HTML and Notion

Absolutely. It's far easier if the thought originated in your own mind.

The default long responses of LLMs don't make it easy.