Item — HN Reader

I stopped using coding agents after more than one and a half year of active use, it really started to become way too boring, and I’m t a point where I just hate having to babysit them and for the 200th time make it understand what the actual goal is… and to be honest, going back to writing code by hand without assistance is really hard at first you continuously have that little voice telling you how simple that would be with an agent. Then after a little bit you’re back to being productive, but I still get that voice in my mind. I’m wondering if that’s how addiction feels (way lighter of course).

That whole experience of going deep for a while into LLM coding, then trying to leave it behind made me pretty pessimistic about the future of our profession. We are creating a whole industry of people delegating their ability to work to a software stack currently controlled by basically 2 companies (that both have very sketchy financials). Doesn’t feel healthy

2 comments

Neither of these points feel true anymore.

Models are very much predictable these days (except anthropic models). The real issue stems from letting them work on their own for far too long. Also we are not controlled by 2 companies anymore as kimi k3, deepseek flash (and soon pro) as the ultra-cheap variants, glm 5.2 especially is a direct replacement for opus 4.8.

Models will only get better and cheaper I wouldn't feel too pessimistic and wouldn't feel too bad on relying on them to accelerate work and free up mental space from menial tasks.

As a personal side-note I never let my agents do architectual design I only use them for implementing. I always found the actual coding part of programming extremely boring and coming up with designs, experimenting and testing the fun part.

I always found that if you are good enough at whittling down boilerplate that coding becomes something akin to pure architecture.

I find that mediocre programmers and LLMs are bad at both. They're helpful if you want to shit out some repetitive boilerplate or perform a complex search of some kind but otherwise you're better off without.

ehh, they're pretty good at automated performance research and bug fixes, especially when spanned across hundreds of them.

bug fixes not so much. when I encounter a bug the first thing I do is ask an LLM and not only do they usually fail to get the correct solution, they often slap virtual duct tape on it which makes other bugs more likely.

feels like a harness issue, bug fixes work out fine with omp.sh as long as it can be tested properly. Untestable things tend to lead to hallucinated results though.

might be a symptom of writing complex code. I'm sure on basic CRUD they can figure out most bugs since most of them have happened 1000 times before.

I doubt it, a lot of what I use llms for is on the very edge of what these models are trained for.

Even if I was paid for using LLMs I would still find the experience nightmarish. I really don’t want to be dependent on such a system and spend my time managing fleets

How exactly will models get cheaper?

Compare the performance of a 980 and a 5050 and I am sure that will answer your question.

Also models baked into the silicon are able to achieve efficiency that is simply impossible to achieve with programmable circuits, there is a general slowdown in the raw capabilities that transformers can achieve and agentic tool use is simply an amplifier that will reach a wall eventually. It wouldn't surprise me if we saw within 5 to 10 years accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s as well as api costs going down to what we already see with subscriptions today.

There has been quite a lot of off-ramping going on where people feel satisfied with the performance they're getting out of the models and simply staying there instead of using SOTA.

  > accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s

how do you update that baked-in model for things that have happened in the last say 2 months?

if i'm a programmer for example, even being a couple months old is a huge annoyance because programming languages and frameworks are changing all the time...

When was the last time you heard about someone talking about "Knowledge Cutoff" dates? OpenAI used to make a huge deal about it every release, now its not even mentioned.

We give agents tools, the ability to read a man page, the ability to use web search. Knowledge cut-off is far less important than it used to be.

I'd be really interested to see all the software that is written by agents. Whenever I touch agents or ai I can't get much use out of them. My understanding is the value when I think aloud with them/treat them as a better google search, but thats about it. Except one off web stuff, that is a pretty neat use case.

But lets be real, anything moderately complex that is out of the domain of publicly available sample code is hit or miss compared to the time invested running the loop. I'd much rather invest the time in myself.

What a lot of people don't talk about is the inherent security nightmare of trusting ai agents and the sheer data exfiltration happening behind the scenes.

Idk, ymmv. I’ve had very good outcomes writing embedded firmware for complex systems, drivers, etc. it’s really, really good at not bodging the kinds of things that always catch meaty developers unawares.

OTOH, we run extensive harness optimization, where everything is specified in advance, then a plan is made, then a naive review of the plan vs the specification vs the blast radius, then implementation, tests, then a naive review of test coverage, a naive review of the code vis a vis our code guidelines, a review for smells, a review for silo violation and architecture compliance, a reconciliation of the documentation, then planning the next subfeature, etc. probably 10 percent code generation, 40 percent documentation and planning and adversarial review, 50 percent automated adversarial code review.

We use a Claude for planning and generation, sol for adversarial review. Our metrics say we are about 2x. Productive, at a cost of about $300 per dev per month.

We are also shipping less bugs and better, more clearly written documentation (we use technical writing English style guides implemented by Claude)

I think a big part is the constant adversarial review by a different model with no prior context except the coding standards. Also important is context management, we do an onboarding and wrap-up for each session where we have a batch of continuity documents- Learnings, musings, and roadtrips where we let the most successful high-context sessions research and then write about something that they “ became curious about” during the session. That actually brings in a lot of insight to the team and occasionally is brilliant.

Also critical is crossing compaction barriers (standard re-onboarding protocols, writing transition documents prior to compaction, etc)

Also critical is being able to smell when a session is going off the rails. What we do there is sideline the session, wait for master to advance a bit, then have it do a “4c’s” review of the committed work, blast radius analysis, and remediation. If there’s still useful context left we put it into a project unrelated to its failure context. That salvages the valuable session context without staying in the failure trench.

It’s a lot like herding cats.

Hi, this really resonated with me and the approach I have been following within enterprise software domain.

Do you mind outlining your stack around this. I know you mentioned python to support your verification harnesses but I am more interested in the agent setup. Are you specifically using Claude and it's skills with custom plugins or are you using other harnesses such as pi. I have settled on superpowers plugin across Claude and Codex, Cursor and most of my time is spent iterating through the design doc between Claude and Codex, implementing with either and starting another review cycle with the implementation, using TDD approach. It can be a lot of work but the end result is more than if I had done it myself. I am trying to formalize more. Anyway, thanks for the great insights.

How much does your whole setup cost? Just curious. An estimated range would be interesting to have

We built all the tooling as Claude-code tools (mostly in python) and the rest is just documentation for preloading the context and as reference material to pull in by policy before certain tasks. Really, there is nothing complicated or heavy, it’s all process.

About $300 per dev per month. Importantly, we are not buying tokens, but the coding plans.

For what it’s worth, I’ve made really good, state of the art software in my areas of interest using LLMs. So I do believe you can produce really good software using them. But it’s domains where I have decade of experience.

But even with that result I don’t think it’s something we should bet the whole industry on, and something I personally don’t feel comfortable relying upon