Item — HN Reader

> LLM speedups have always been predicated on not understanding the code they produce as well as if you wrote it yourself. Reviewing even well written code to understand it in depth, has always taken longer than simply writing it yourself - let alone sloppy LLM code that you have to fix

I don't think this personal opinion holds well when contrasted with reality. The truth of the matter is that in moderately large codebases, which is the norm in any production setting, at best you have a T-shape understanding of the system: in specific areas you might have somewhat deep understanding, but for the vast majority of the system you have at best a high level understanding of the software architecture.

This is why software engineering frames things like design patterns, principle of least surprise,single responsibility, etc as premium design and code quality traits: they allow developers to effectively extrapolate their high level understanding of the project onto components they know nothing about.

In any software development setting,you will find that the most successful and even senior engineer is that which successfully and skillfully manages this uncertainty, and is able to hit the ground running on environments they never touched. I had a colleague who once was a senior software development engineer at a FANG that mastered this art, and called it JIT onboarding.

So why are you pretending this is something new or novel?

I think this blend of criticism only serves to allow inexperienced developers to stand out by complaining that a very mundane aspect of working on projects as a member of a sizeable team is somehow a novel development caused by LLMs.

And don't get me started on the "slop" nonsense. Have you actually browsed through code written by people?

3 comments

Thank you for writing exactly why people who complain about LLMs making it harder to understand what’s going on are so off. Nothing helps more than asking LLMs to tell you exactly what’s going on! They are insanely good at that. I wrote the plugin system in my product, but many people added to it over a decade and today I have only high level knowledge of it. But if I need to know something about a very specific part of it, I ask Claude and now I know exactly what I needed to know. I could even write the code by hand now, and sometimes I do for the fun of it, but Claude has just explained to me, so it also knows exactly how to do it so why not let it do it?? I can understand as much as I need to by reading the code , asking questions, carefully analyzing the tests, much more than I would if I had needed to focus on every single detail that is irrelevant to the task. Sometimes I even do some minor refactoring just to try and feel like I’m “inside” the code as I used to. But I know I will likely be out of the loop again in 3 or 4 years, as I just can’t remember everything I ever did. But now I have a LLM ready to help me get into it again whenever needed again.

>So why are you pretending this is something new or novel?

Because as the OP of this thread and LLM users are rediscovering, that understanding is the most critical part of software development

>I don't think this personal opinion holds well when contrasted with reality. The truth of the matter is that in moderately large codebases, which is the norm in any production setting, at best you have a T-shape understanding of the system: in specific areas you might have somewhat deep understanding, but for the vast majority of the system you have at best a high level understanding of the software architecture.

I suspect a lot of people haven't worked on a system with engineers who've been there for 20 years working on it. There's simply no substitute for that kind of deep institutional knowledge, and it'll take you 5-10 years to be as productive as them simply because of that level of extreme knowledge about a codebase. It isn't as sexy as swapping jobs every 2 years though, so you have to go outside of the big tech bubble

I will say that yes, sadly, I have browsed through code written by people, and sadly also have browsed through code written by LLMs. Who do you suppose wrote the code the LLM was trained on? Divinity?

I'm being tongue and cheek, but as someone who uses these things extensively in work and personal projects, I want to point out that it's new or novel that generating lots of plausible code is now vastly cheaper than it once was, so the scales are quite different.

And yes, slop is an apt moniker for something that requires lots of careful prompting and configuration and guardrails (which is extremely sensitive to small changes in these initial conditions in ways that can be fairly nonobvious) to produce something that will in most cases read at least a little bit worse than if a human had written it. And it will still go off the rails sometimes!

To be clear I don't really agree that typing out the code generated by LLMs is a reasonable counterweight, it largely defeats the purpose. I agree that at a certain point you have to treat parts of your code base like a black box and enforce modularity. But this is still nontrivial, especially in more specialized domains with lots of nuance

> I will say that yes, sadly, I have browsed through code written by people, and sadly also have browsed through code written by LLMs. Who do you suppose wrote the code the LLM was trained on? Divinity?

Training is a red herring. The root cause is constraints, or lack thereof. If you lock a junior dev in a basement and force him to deliver the same features that these LLMs output, you will see exactly the same type of slop. The root cause is that junior devs are inexperienced and oblivious to best practices and guidelines and even the team's internal standards. They output code unconstrained by these guidelines and thus output big balls of mud.

> I'm being tongue and cheek, but as someone who uses these things extensively in work and personal projects, I want to point out that it's new or novel that generating lots of plausible code is now vastly cheaper than it once was, so the scales are quite different.

I agree, but we need to be mindful of what is the actual root cause. I argue it's not AI coding agente but the diverse source of these code changes, which we also see in production settings in projects managed by large teams. In teams manned by junior devs you see very much the same slop building up to a big ball of mud in a few iterations. This is nothing new. What changed is that now ai coding assistants grant everyone access to what amounts to a large team of junior devs.