Item — HN Reader

That quote is the lead in to an example I talk about after this quote illustrates what I often see.

> Agents bias to making the current change as safely as possible. I had a situation in a previous codebase where one morning, pre-caffeinated, my meat brain mentioned using browser local storage. So some random state was managed in local storage. Everything else through a backend database. When I looked at the code, the amount of wrapping and indirection to preserve this idiotic human mistake probably tripled the LoC. Agents can amplify our one-off bad decisions by being so conservative.

You can of course solve this many ways. And many of boils down to just how a particular humans brain works. Some will solve this by not reading code. Some will read / write code.

Whatever works for you is great. But many there is upside to the precision of not having code intermediated through the LLM for many.

1 comments

Right, but this just seems like underspecification. In my experience as both a team leader and an "agentic engineer" (ugh), I try to blame myself for the lack of clarity of my asks, rather than the person/agent for making the "wrong" choice.

I'm sure plenty of meat humans out there would make the same mistake (sorry, you said to use local storage boss!). You might give them a scolding. And maybe document that policy. Maybe in a markdown file for the next person. IME the latest models are significantly better than the median engineer at following this feedback.

I don't think it's fruitful to blame the LLM any more than it is to blame someone working under you.

In fact I would say this is an excellent example of how engineering does NOT fundamentally change in the era of AI.

Often times to write a good enough specification in a way the LLM won’t misunderstand it, you need to really understand the code and the possible pitfalls in it though. A few times when I tried this route, even adding details just in case the LLM still went and assumed something wrong that I didn’t consider would be a problem until the agent had to make that choice. Then it becomes a game of doing it again and again with more and more detailed instructions until you could have yourself done it in less time and having a deeper understanding of the problem (or I could argue having any understanding of the problem at all, since having the LLM do it for you is like being a manager who has no clue about the lower level decisions made and only being communicated the final result).

Yes but I always have to be on the lookout for this meta pattern that leads to code bloat.

In this case things mostly sorta worked and the simplest way to see the problem was look at the code. And try to take it apart a bit to see where the problem was.

I felt I arrived at a better pattern I could trust that the agent could use much more efficiently this way than asking the agent to do it. I could then test that the pattern was being adhered to and therefore better trust the agent not to go off the rails.

I personally internalized the details a lot better by doing this writing. I wouldn’t have internalized it - or more likely played whack-a-mole - by guiding an agent.

How do I arrive at the patterns to check for without exploring the code? And capturing a real failure case?

> Yes but I always have to be on the lookout for this meta pattern that leads to code bloat.

You don't!

Have you tried adding rules/automations that make it explicit to review and fix the code for bloat (per your taste, with examples if needed)?

With this setup + a good frontier model you will never have to be on the lookout for code bloat. You can even get the agent to send you text message with the LOC-- if it makes you feel better.

I think I understand where you're coming from, that it's hard to "let go" (I've been coding for 30 years and it was hard for me). That's why I'm recommending to have agents write verifiable quantified reports of the things you care about, so you can build up some trust in the agent's work and you don't have to do things by faith.

How do you arrive at a failure case to measure without looking at the code?

How do you establish sane patterns when you’re in an exploratory /architectural mode? You could do this in English, sure, but many of us do this more efficiently and precisely in code in a way that lets us be careful, internalize details, and add measurement + QA to ensure it’s adhered to by agents or humans.

I’m not saying to write all the code. I’m saying it’s useful to write 5% of it. Then let an Agent stamp out / rewrite the rest

> I’m not saying to write all the code. I’m saying it’s useful to write 5% of it. Then let an Agent stamp out / rewrite the rest

100% agreed. And it leads to far better LLM output and lower token usage, too, I find.

>I try to blame myself for the lack of clarity of my asks, rather than the person/agent for making the "wrong" choice.

I noticed this phenomenon way before LLMs came along. Some people would put a halo around a particular technology and blame everything around it when something went wrong.