The Coming Loop

Loops work when you spend the proper amount of time to understand what you want ahead of time. The prerequisite is clarity — enough clarity that you could write a careful specification that you could hand off to a junior colleague.

Often, it takes 5-6 broken crappy versions of a thing until you understand that. There is no accelerating the 5-6 broken crappy versions - there’s no agent tech that’s going to help your meat brain avoid thinking time.

So most of my time is iterating between these two phases: I don’t understand what I want, I need to read and write and play with code, okay it’s been long enough I think I know what I want (it is extremely easy to deceive yourself) … okay now I do actually know what I want and I can write a loop.

Many people think they can jump ahead with agents. You cannot fake understanding or clarity. It is painfully obviously when someone skipped that meat brain understanding phase.

> We may create codebases that are not merely hard to maintain by humans, but that assume machine participation as part of their maintenance model... People more and more merge code they cannot fully explain. People lose their ability to create issue reports or discuss things in chat, without augmenting or rephrasing their messages with the context provided by a clanker. Too many people increasingly rely on a machine to summarize or contextualize it. More and more do I encounter people who converse with me through the indirection of an LLM.

I experience this daily now. It find it discouraging and concerning.

I believe we're merging more code we can't fully explain because we are now relying on code review to build the mental model that was previously built by writing code and collaborative technical planning. I don't think code review is fit for this purpose. I do think we can extend code review with structured exercises, informed by pedagogy, that strike a better balance between friction and understanding. (I'm looking for help testing these exercises).

>Yet even with a lot of manual steering, that type of code does not come out of LLMs naturally, and even if the code comes out naturally like that, they will still attempt to handle now impossible errors.

This is something I’ve struggled to fight against in many PR reviews. Especially once already written, convincing someone that their excessive null checking is harmful is an uphill battle. Short of better modeling (and languages that allow for sum types to enable it), I haven’t been able to come up with a universally convincing argument against this kind of “shotgun parsing.”

Maybe it really just isn’t that big of a deal? But when actually reading through and refactoring a codebase I’ve always found it frustrating to manage these unnecessary checks. Sometimes they’re nearly impossible to delete safely once present without first adding some kind of logging or broad investigation.

This ties into something I have been saying for months: LLMs are great at finishing tasks, but bad at aesthetics and taste.

There are two kinds of work: One is goal-driven work, where we have a goal to achieve, and we care very little about how we get there. Security is a perfect example; if you want to exploit a system, you rarely care about how beautiful the exploit is, all you want is access to those super secret nuclear plans. Research is also like this; "research-quality" code was famously terrible, even before the age of AI.

The other kind of work is taste-driven work. People think that, when they're adding a feature to a large codebase, their goal is to add that feature, but that is often not the case. Keeping the codebase amenable to future changes is often far, far more important than this specific feature, and that requires taste. Note that maintainability and code quality aren't synonymous, code quality is just a means to an end, and that end is maintainability.

My experience is that I am bottle-necked on specs. The agent loop is less of a thing for me now.

If I can get a clear understanding of what I want to build, communicate that to Claude Code in planning mode with the goal to write an actionable spec (not code, plan to write the spec) then I tend to get very good results once the agent goes to implement.

But this strategy, while effective, puts a big load on me to write the specs. The agent tends to knock each one out of the park (usually 2 to 3 follow ups based on code review) but then I'm back at the stage that requires the spec.

Another issue for me is that when I step away, if the agent finishes a task and could technically start on an existing spec (no overlap on files so no conflict possible) it doesn't know it can just create a new branch and start. Before I go to bed I'll often say "do task X and once done and pushed start on task Y". But I haven't had luck beyond that. Often I find that it starts on Y and has a question and then the agent is idle the rest of the time.

The final issue is dependency coupled with the above. For example, today I was writing a background job processor. Obviously, the jobs that are in subsequent tasks require the system. That happens with some frequency. Even the specs need to be refreshed after the implementation to take any details that were resolved at coding time into account.

But I am just on the cusp of wanting the outer loop. The gate is almost entirely on spec creation and PR review. In places where those gates don't matter, I want the agent to keep chugging away.

As an aside, I strongly believe we need to start using tools that are better for LLMs even if they are worse for us. For example, Rust is annoying because the compiler is so strict. Bad for me, great for LLMs.

Code is part of a shared and built understanding of an information system.

If these loopers mean we all have to move at this continuous wave of software happening, then we get to the highest levels of logical information system design and its all human judgement and balancing of business requirements to fit a given niche in a company or market. So all the programmers have to become business analysts/market researchers/businessmen...except the specific niches where AI tooling can't really clank well...or the end of the subsidized AI token era makes all this looping too expensive to continue. This feels like expert systems and symbolics lisps machines redux, where we briefly ran into the fact that its not so much the code itself not being able to do stuff, it's that your company's org always gets shipped, so if you can't change your company org, your software only has so much flexibility.

Dataflow diagrams and domain knowledge / domain modeling / ubiquitous languages may become the metalanguage that we start to use and set the standards for quality, functional, and non-functional standards and conventions. We make the "looper clankers" ensure that they fulfill that data / behavior / performance contracts before saying what "done" is, because "done" is no longer just code that compiles, code that builds, code that deploys, or even code that sits in production; it's code that fulfills all of the user requirements, operator requirements, and maintainer requirements. So, the language used may be required to make us all turn into business analysts and software architects more than syntax knowers. The revenge of UML and the return of declarative / logical design / BDD triumphing?

(Typo scan by gemma4-12b but I didn't let it alter my message)

> the right fix is not "handle every malformed case." ... [LLMs] will still attempt to handle now impossible errors.

This is the number one code smell from LLMs and I don't know why they are so obsessed with it. In python, it often comes as `hasattr` checks on types that are defined to have that attribute, in a code base that is fully type-checked.

Why do they do that? Is it from pre-training or re-enforcement? If that latter, can the labs please fix this?

I keep thinking about at which point I should not force myself into the loop. As a developer I really like working on the code structure, making it clearer, thinking about good abstraction, breaking into modules, etc. I really take pleasure in it. At the same time I understand that at some point I am becoming the limiting factor.

If the point of the software is benefit people, should I still care about how the code looks.

Right now, I still think that the answer is yes, but in 3 years? in 10 years?

> My current status is that I have not had much success with this way of working for code I deeply care about

If something is judgement heavy, "code i care deeply about", then i don't really agree with the direction of travel here. Don't try to delegate decisions you care deeply about.

I do like the framing of agent loop vs harness loop, but only delegate stuff that you can accurately specify in advance, that usually means stuff that's repeatable in my case ("hey go see how i did X, do that but for Y"), and that inherently means stuff that's predictable.

For stuff where lack of my judgement as input is just going to cause me to say "no", we're down to collaborating in the "agent loop" as Armin puts it. And that's totally fine. It's fast, but also safe.

Remember before AI coding assistants, sometimes you'd get an engineer join your team who was SUPER productive, your peers would be jealous "oh yeah but you guys only got all that done because you have X on your team!" - they didn't live the curse of having that kind of person around - if you don't have them PERFECTLY aligned, then they run off at break neck speed in the wrong direction.

I think what is going to happen is revival of "Methodology".

"Methodology" was a big thing in the past just before we got into "Agile Extreme Coding", instead of trying to model the big picture of SW development projects just jump into coding agilly. Implement it feature-by-feature

Granted the methdologies proposed ( See: https://www.ibm.com/docs/en/rational-soft-arch/9.7.0?topic=m... ) may have been too heavy and not flexible and not improved enough. But now with the rise of Agents I think we need to revise and perhaps re-invent them for AI agentic development.

Thank you for writing this thoughtful post, Armin. I find it deeply comforting that the developer of Pi, an agent harness, does not remove himself from the loop, like me. Maybe if I started thinking of codebases as biological organisms I could get comfortable with getting the human out of the loop.

What does any of that mean in practice? it's just rambling about abstract concepts that seem to be designed to hint at a bigger picture, when it's just getting AI to write code for you.

Is this where it's going? Having to mystify our roles so it seems like we're still the thought leaders when actually we're just becoming pseudo-teachers that try and herd our group of AI idiots to the right conclusion for us so we don't have to, without ever giving away that it's just all techno-babble?

We used a “loop” before it was called that to drive MS-DOC support into Tritium. Based on that experience, I take issue with this:

“There are already impressive examples of large automatic porting efforts, including the reported work around moving parts of Bun from Zig to Rust.” (Emphasis added.)

It will be impressive if/when the Bun team is able to pick up and continue extending and supporting Bun. For us, MS-DOC remains read-only and probably perpetually buggy until we reimplement with a better understanding. Until then, it’s definitely not “impressive”. Functional? Maybe. Impressive, no.

I am 100% for fully agentic loops... for tasks other than engineering.

I'm not willing to outsource the understanding how things work part of myself. That part of myself is what got me into computing in the first place.

If this work becomes simply a matter of describing intent to a machine (probably through an Issue, like a user), and going to check on the result when you get the 'done' notification: I'm done.

It's possible to use the tools to do awesome things without letting go of full system understanding of the parts that you look after.

I'm a software developer from way back, using tools and languages that coding agents are far less familiar with.

So when I use an agent to write code, it's in languages I'm less familiar with, and often using libraries I know nothing about.

All to say, my part of the process often ends up being:

1. "Here's what I'm looking for, in detail" 2. "That's not right. Here's one way it's not right, and a specific example. Please fix that." 3. Sometimes I give suggestions for how what is going wrong might be happening, or conceptually how to work around the issue. 4. And iterate on 2-3 until the result is close enough.

That's a loop I'd love to automate.

This.

> Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.

Context-smithing can help to a degree and cyclomatic-like complexity rules tend to make matters worse. So, you either roll up your sleeves or close your eyes and hope for the best. I've had limited success with the latter.

> You Cannot Quite Opt Out

I am so over this. I cannot take anyone seriously that claims inevitability of their ideas, and how you must adopt them without "being left behind". If these tools are so good and so capable the result should be able to speak for themselves rather than this FOMO inducing, emotional language.

> Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.

It’s almost as though these models were trained on a vast corpus of largely mediocre code. They will never outperform the median Github user - it is all they know, it is all they can do.

I’m having awesome success working with recursive agents. I discussed my experience with them. [0]

> Claude's attention doesn't distinguish between "instructions I'm writing" and "instructions I'm following" -- they're both just tokens in context.

It takes a little human help in the first iterations but after a while it will start to iterate and improve unsupervised.

[0] https://github.com/adam-s/agent-tuning

Show me the billion dollar solopreneur startup, or the profit increase for companies and at that point I’ll start thinking that this tasteless high level wanking might make sense in some way

For some reason pro-ai blog posts feel like paid ads, I might be wrong.

Part of the problem is that models don't have a strong sense of taste, part of the problem is that the context in which projects exist is incompletely represented in the LLM context, and part of the problem is that LLMs tend to be myopic.

The lack of taste can be mitigated to some degree by improved training, though taste is not a stationary distribution in humans (see trends/fads/etc), we can at least better track the cutting edge. I think this area still has low hanging fruit but frontier labs are more concerned with being able to solve problems than the style of the solution right now (for evidence of this just look at the Opus 4.5 -> 4.8 arc).

The problem of incomplete context is partly a human problem and partly a harness/interconnectivity problem.

LLM Myopia is a harder problem to solve just by virtue training models on question/answer pairs. Countering this requires emphasizing RL on solution paths rather than just prompt/response, which is doable but harder.

One thing that I'm certain of, is that there are always costs to writing more lines than what is necessary to solve a specific problem.

I've experienced this before AI, and I've experienced this, magnified, with AI.

For a well-designed project written by hand, you will work faster writing the code for new features by hand than the same project written with AI from scratch, using AI to write the new features.

But... If you have a well designed, human-written codebase, you will be faster if you generate the code for new features using AI than if you do it by hand... And you can maintain that speed for long periods of time if you use fine-grained prompts. What matters most is the quality of the codebase.

You can achieve the same degree of maintainability using AI from the beginning but you would have to make fine-grained prompts.

The gap is about making good engineering decisions. It just so happens that the value of good decisions compounds over time.

I have huge respect for Armin but all of the concerns about agents producing more code with less competent supervision from senior engineers doesn't seem that different from the status quo to me. A vast majority of all software I've ever professionally worked with has been terribly structured, hard to work with, full of bugs, etc, produced by mediocre to bad engineers and run by semi technical product owners or managers who basically promot the software into existence by making jira tickets on 2 week cycles to hold it together.

Yes it's awful, but it kind of works and has worked for a very long time. Agents are already improving a lot of open source software. Yes they're producing a lot of slop too, but having beautiful code, understanding how the system works and being able to delegate to a competent engineer you trust is reserved for the very few right now and I think we have all the systems and experience in place to deal with "bad" but working software so personally I am not concerned

Theres a deep insight in this post about the value of looping for throw away code to explore a problem space, rather than brute force a problem by just applying more tokens and hoping.

The more I play in this space, the more I’m drawn to the idea that some kind of back tracking constraint solver is a better solution than then the current naive while loop / brute force approach here.

The results I see are similar to what you get from a greedy brute force constraint solver; solves trivial problems, sometimes solves harder problems after a long time, takes too long to solve really hard problems; solutions are increasingly non optimal on average as complexity goes up.

We have so much existing knowledge about building good constraint solvers, if we could just figure out how to apply it here somehow.

We've had great success with agents thus far at my job. A year into Clauding and all our dev metrics are up while our downtime has remained steady.

Being an iOS engineer, much of my engineering cycle these days is going from Figma/PRD → spec → code. After being handed off to QA, we handle the bugs and product slips as they come through, while we simultaneously build/spec the upcoming addition. This is basically the same agile style that's been popular for 20y, just super-powered with agents.

How might someone accomplish the same goals using loops instead?

I have had some success with /goal for long tasks that can be set up in a way that the agent can do good work for an extended period of time.

A lot of tasks aren't amenable to that, and the ones that are still need a lot of care to be set up correctly. The default vibe coded codebase won't be.

I've come to think of the activity of choosing the right technology, the right architecture, the right testing setup, the right context, and the right /goals to use as programming the agent.

There's a major problem with evaluating the result of agentic coding, and "loop engineering" exacerbates it.

Like many crafts (painting, music, etc) the true test of great software is time. Not because of taste, but because software is a living artefact that must evolve.

Only over time does a human developer realise they have "painted themselves into a corner" and chosen the wrong abstractions. And it will be the same with vibe coding (i.e. where code is not architected and reviewed exhaustively by a human).

Because companies won't surface their internal codebases, the most likely signs of such problems with be the reliability of applications and services. I don't know if recent outages are indicative of changes in code practice, or just coincidence, but we'll find out over time.

Agents aren't yet strong general reasoners - they are good at reasoning in the shape of their training corpus - so it's essential that important code is reviewed by human experts.

>>> For now I have not moved past the point of comprehension being important to me.

I see software as new form of literacy, even in the AI world, so yeah in my world view, comprehension will be something we always cling to.

I might comprehend some code the way I comprehend the newspaper article on the second page, others I comprehend like a Dylan Thomas poem. My attention might be different but I still need to understand it.

Quoting the creator of CC holds little value in my opinion. I too call my product good.

> opting out of this fully machine-driven future may not be an option.

I am contemplating whether I want to stay inside this rat race.

I completely agree with the conclusion of this blog post, by the way. I feel uneasy, and I do not enjoy the work I deliver using LLMs. I think OP did a really good job on capturing at least my current state.

I can't help but be tired of the LLM trendy, where people bang at loops until they hope the model sculpts something. It feels so empty mentally to just have results without constructing it.

That said the idea of loop has always been there (iteration, V cycle etc) but I'd be glad to find people with more theory and less agents swinging blindly so to speak.

Reading the comments here, I'm struck by how many people seem to view code quality as an optional thing. It's almost as if software can function correctly without it. But it can't. Code quality is not a nice-to-have, it's a must-have. For any moderately-complex offering. There's a certain level of complexity that you can never reach without a well designed codebase. Cannot reach no matter how many humans or AI agents you throw at the problem.

The idea that it's a 'nice-to-have' is an illusion. It's like if you borrowed a lot of money to fund your startup and you still have some cash in the bank; at that point, a viable business model might seem like a 'nice to have'.

It's only when the lender comes knocking and you don't have enough money to pay them that a viable business model suddenly becomes a 'must-have'.

The author is spot on about the paradigm change of software as a lifeform. Living things provide us with genuine interactions and experiences of learning and growing, without forcing us to understand the code - You can learn to work with animals and plants without understanding their genetics at all. I believe this is how our relationship with software must develop, and in order to get there, we'll need to learn to design and develop software in a completely new way. I've been testing this hypothesis in my spare time, hacking together a server-browser system I call Mycelium. It's a bit like OpenClaw, except you can use it to create private local Webs, and print custom 2D Electron browsers to view and work in these webs.

I feel like what should have happened with AI is that teams should have started to put more effort into planning and pre-implementation discussions. Second thing, team leads should have felt more comfortable to reject large Pull Requests.

When I see a large 10k-lines PR, I feel a sense of panic. In a corporate setting, I also feel a kind of pressure to approve; the more work was done, the more pressure there is to approve the PR. This is why I think up-front pre-implementation discussion and alignment has become essential.

You really can't have people going rogue and weaponizing their AI-generated lines to gain control of a project through the duality of brittleness + complexity.

Brittleness + Complexity = Control

The most important part of working with agents is removing tools, especially tools they like to use. Do not let them use cc, make, git etc until it's a necessity 90% of changes introduce a bug? ok you can start using gcc/clang/rustc whatever. Tools generate too many tokens, and it can't be fixed by '| grep' since they just start spamming cli calls.

At least this is true for antigravity it tried to used git a few times and I gave in and then I discovered there was a token limit. Velocity plummeted. If you want git before it becomes a requirement best to mirror the directory into a separate git repo.

Generally interesting reflections here, yet I see the same kind of myopia and fatalism that is rampant in our (fashion) industry:

> yet I have no doubts that this looping future is going to be our future despite the fact that I presently resent it

Why would anyone concluded this? LLMs are just one kind of application of MLs to software production. There is a vast solution space for automating parts of software production. The idea that slop loops are the inevitable future because they happen to be accelerating output at the moment just seems profoundly short-sighted and lacking in vision.

A friendly reminder to just do 9 to 5 and touch lots of grass. None of this shit represents industry trends, majority of people still use chat interfaces and copy blocks of code. There’s zero early adopter advantage here, only FOMO and lots of anxiety.

The issue is that whilst the loops will initially lead to good results they will be less and less as context gets bigger and bigger and tougher to understand for human and AI.

So it depends really on the size of your project.

I think a lot of people here have either not read the article fully or are misapprehending it.

Neither this author nor most other sane people I know claim that the code or architecture these "loops" produce is great. In fact, the author explains how it is not great. His point is rather, that we'll increasingly see a world in which code quality and maintainability by humans will cease to matter for a lot of codebases.

There might be many software companies in the future which successfully sell software products which were created without a single software developer being involved in its development or maintenance. The code might be bloated and bad - but it doesn't matter because machines can still create and maintain it cheaper and faster than people can.

I already see this happening at a small scale at the place where I work. Product managers with zero coding ability are attempting to create entire new product features on their own using Claude or Codex. We do not let them merge this stuff unsupervised but in some corners and in new repositories they are publishing stuff that they have barely spoken about with a developer. They are just doing it. We'll see more of that.

I just tell it "you have until morning to work on this, be careful not to use too much ram and don't burn the cpu"

and then it goes off to do its thing and hopefully rngesus is with us.