back
446 comments
I've hit this point with AI where it's not a simple process, but a long drawn out back and forth.

I'll use AI to design the implementation of a medium sized, cross cutting feature. Review all the details, maybe iterate on just that. Then implement with Claude 4.7 Max - which runs slower, but does a better job. Then review the implementation, then have Codex GPT 5.5 xhigh fast review it - which almost always finds corner cases. Have Claude fix those - Claude is better at writing intuitive maintainable code versus Codex overengineered/shortcut filled code. (Codex is better at finding/fixing bugs and doing reviews - it's annoyingly pedantic)

Then repeat with fresh Claude/Codex instances having them both review the current staged changes and getting feedback, handling the feedback. Then covering it in tests. I mean overall I still implement the feature faster than coding it manually, but I spend a majority of the time going back and forth with reviews, handling corner cases and at the finish end up with what I feel a really solid implementation of whatever feature I'm working on. The v1 feature feels more like a v3 given the amount of iteration it already went through.

Talking the problem to death with the AI before implementation is a nice zone for me. I feel productive, get good results out of the AI, and still largely understand the code. That’s the part of the AI revolution that I feel has made me a better engineer because I argue about design and architecture all day with a robot.
yes exactly. Too many people ask AI to one-shot complex tasks, and wonder it behaves like a junior asked to rush something.

I have my own skill: 5 rounds of research/planning/test-planning. Interactive with me in loop for all important decisions. Starts with high level shape, then details. Planning can take 2-3 days of my time, then the implementation agent can take many hours (Opus 4.7). It splits the implementation across many phases/commits, each with its own code-review fix loop. Deep code review at the end can take another hour or two. It opens a PR, Gemini reviews, it reads out and resolves those issues.

Projects still take days or weeks, but 5x faster than doing it all myself.

Edit: the skill - https://github.com/scosman/vibe-crafting

When I use ai to code this is pretty close to my workflow too but I find it ends up taking at best just as long as if I were to write the code myself. If m some cases I’ve thrown away what the ai has done and just done it myself. I think that’s just a skill people need to learn - at a certain point you have to cut your losses. I’ve seen some coworkers argue back and forth with an llm trying to get it to do something. Especially true on simpler changes.
That sounds too much like three weeks of work saving you three hours of planning.

In my experience, software engineering is a matter of knowledge. Understanding it and then coming up with a solution. The latter is a flash of insight that comes mostly from experience. Then you gather more information to flesh it out, or brainstorm it with your colleagues.

What you're describing sounds more like a ritual of doing busy work than anything practical. Because tasks vary so much. A feature may be huge, but you take care of it in a day with copy pasting because you already have all the building blocks in other files. And something may be twenty lines of code, but you spent the whole week sweating on it (concurrency stuff maybe). Those ritualistic workflows sounds more like someone imagining software development than actually doing it.

And then Anthropic has an outage and you what...have a coffee break until then? All that time babysitting the AIs just to be a little faster but probably with less knowledge/control over what they did?
This all sounds insane. If it requires so much back and forth with the AI why on earth wouldn't you just write the code yourself? At least then you build the mental model of the code and keep your brain healthy. Reading the comments in here about all the hoops people are having to jump through just to do the same thing they were doing a year ago without AI... and spending a fortune to do it! I think you've all got AI psychosis.
I am not switching the different LLMs as much, but my approach is similar:

1. I write a list of things I want to have without AI support

2. I discuss the list with an LLM, which occasionally reveals obviously missing things I hadn't thought about or just things that would be smart to have. Or sometimes the LLM doesn't get it and wants to funnel me down a commonly walked path, which is a non-goal

3. From that list I draft an implementation plan containing things like how the code shall be structured, which language, libraries, build systems, etc to use. This may even contain some data models and considerations that are more detailed, like for example ideas about how a specific interaction shall be event sourced. I work on that, till I feel a satisfactory level of clarity has been reached

4. Actual writing of code as a back and forth between manual writing, letting an LLM write something and so on. LLMs suck at writing CSS that feels like good UX design to me, so usually templates, layout and CSS will be (re)written entirely by hand

5. Bug-hunting and guessing potential edge cases is one thing where LLMs really shine. Often if the work before that was quality the LLM has an okay time coming up with fixes that are no worse than what I would have done.

Similar approach, but I also go a step further with some basic manual architecture/high level contract/stubs setups, just to keep it consistent with other systems (and easier reading as well).
How much are you spending a day for the tokens to do that?

Ingest big project, comment on it gets expensive. I'm not sure how expensive.

This article doesn't address writing code with AI, just code review. My issue with agentic coding is that I make numerous micro-architectural decisions while programming. I almost never have a full spec up front and develop one as I consider what I am writing.

When using Claude Code or Codex, that is all gone. Claude Code is extremely eager to reach the end goal to the point that it feels like a fever dream to write code with it. In the end, I have low confidence about edge cases and fit into the project's architectural and design goals.

On top of that, I enjoy programming, reverse engineering, etc. and I feel that the LLMs, while able to solve some problems or deliver some features, take that fun away. I'm trying really hard to find a workflow with them that I'm confident in, but I fear that workflow is just chat, search, and being a rubber duck for my thoughts.

I find myself spending on average more time in LLM review/resolution loops than it would take for me to write the code by hand. Partially because once I'm in the flow I write very very quickly and the code pours out sometimes faster than I can write. But also because the LLM code on the first few tries is generally really really bad. What I find interesting though is that spending the time to personally review and direct the LLM through several iterations of review and revision on average results in higher quality code written in about the same time as I would have written it. This might be particular to me, but seeing several interations of someone else's code helps me better understand holistically my objective as opposed to whatever happens to come out of my flow-state consciousness.
The linked article about getting LLMs to critique each others' code review[1], the magpie tool[2], and also this recent article from Cloudflare about their code review stack[3] are all quite compelling.

I'm fairly AI-skeptical not on grounds of "do they work" but "are they good for the world". I feel that getting AIs to do this kind of review work is a rare case that doesn't outsource thinking and deskill workers. It doesn't trigger the same alarm bells as having the AI write the code (including having the AI fix the issues it discovers). That's setting aside environmental and other ethical concerns, which are still significant to me.

I have been impressed by the recent quality of AI code reviews*, but the experience of interacting with 3 separate AI reviewers via GitHub PRs is pretty terrible. Having more local-oriented and jj/rebase-aware review rounds would be great.

*context: fairly large PHP/Laravel backend and Vue frontend

[1]: https://milvus.io/blog/ai-code-review-gets-better-when-model...

[2]: https://github.com/liliu-z/magpie

[3]: https://blog.cloudflare.com/ai-code-review/

As a junior, i do actually enjoy going back and forth with the AI discussing different ways to implement something and exploring alternatives.

More often than not, I'd have an architectural idea that I'm not that confident in. The process of talking with the LLM takes a long time but it helps me sharpen the initial approach or even come up with a new one depending on the requirements.

“A lot of people seem convinced that the point of AI coding is to write low-quality code as fast as possible.”

A lot of people think a lot of things, but I don’t think the majority of people think the point of using LLMs is so they can produce low-quality code. Do they produce low-quality code sometimes or often? Of course. But they also produce high-quality code very often. And sometimes they just a “fine” job.

One of the promises - and there are plenty of cases where it’s met and where it falls drastically short - is that agentic coding tools can help us code faster that is just as good or better than what a human can. One of the other big ideal payoffs is that agentic coding can allow non-programmers to create things that previously required programmers to create.

We can debate as to how successful we’ve been toward the two goals above, but I think it’s misguided to say that the majority of people think LLMs should produce lower quality code.

Regardless of what model you use, agentic coding tools are indeed pretty good at finding issues if you target them a bit. And they have no respect for their own code or any sense of shame. So, you can just point them at their own code with a new thread.

Many AI models seem biased to cutting corners by default when generating code, even when you ask them not to. But a few simple follow up prompts can address that. Simply ask for covering corner cases with tests, test all the known non happy paths, look for weaknesses, verify adherence to SOLID principles, do security audits, etc. It will find issues. With bigger projects, you can actually make it file those issues in gh with labels and priorities. And then you can make it iterate on fixing issues with separate PRs.

On a recent project, I made it implement a simple benchmark test for measuring throughput. I had a hunch it was doing very sub optimal things. I then asked it to look for potential performance bottlenecks and use the benchmark to verify improvements. At that point I already had a lot of end to end tests to verify correctness. So, these performance tweaks were relatively low risk. I got about two orders of magnitude improvement and a lot more graceful behavior when pushed to the limit.

If you have a bit of experience engineering systems, just treat these tools like they are junior developers. Competent but likely to skip some essential steps. So, just double check with a lot pointed questions "did you do X? If not, do it now". Anything that needs repeated asking, turn it into a guard rail / skill.

There's a bit of effort and skill involved with this. I imagine a lot of less experienced developers might struggle to get good results because they aren't asking for the right things.

We may be in the last Golden age of AI, where experienced professionals still exist who can code manually, and AI already exists who can code automatically, and when the former use the latter skillfully, wonders happen. This magical intersection may not exist iin the future, or become very rare.
Title of this article suggested more depth and I was expecting actual code examples. But it is like other opinion pieces. It suggests a prompt (ask AI to find bugs) that works for the author advising everyone to do it that way.

I use these tools at both work and for personal side projects and I was expecting to watch and learn. But these opinion pieces without examples are way too many now.

One thing that's been interesting to me over the last few years is charting the edge of my coding laziness. As a coder, I'm lazy about boilerplate code -- I hate writing it, I hate maintaining it, etc. And so I design and architect (or used to) around that preference. Sometimes that's smart, sometimes that's not. But it was my preference, and I avoided something that was hard for me to do.

When LLMs started being somewhat useful for coding a few years ago, and I found they were in fact great at boilerplate, in fact pretty much only good at boilerplate ca 2023 or so, it got me thinking about all the accommodations we make in design and systems architecture that are sort of tacitly understanding who we're working with and their strengths and weaknesses.

The modern models have their own very different strengths and weaknesses compared to humans, and deploying them is a really interesting exercise of different architectural and engineering skills. I've enjoyed it, and hope I continue to.

I’ve landed on a very similar usage in my last pet project. I’ve used the llm mainly as a glorified refactoring tool/LSP/rubber duck. I can define custom skills that act as specific passes over the codebase that are hard to do with traditional tools, I am using Julia, so I have a skill that is only about doing a semantic and type analysis pass to catch potential type instabilities. Or another that is just about documentation reporting. The workflow for me is always: talk the problem to death/get a report. Triage, decide what I can and should do on my own, what can be left to the llm as mundane boring refactoring tasks, what instead needs me to figure out the correct shape first and then ask the llm to propagate the new pattern in the codebase. Then act. A lot of the time I am implementing the llm suggestion by hand on my own to get a feel of how the codebase is shifting under my feet and stay on top of things. This indeed makes things more slow, but allows for an overall higher quality codebase. Especially the refactoring part.
Each time a new technology comes around, we seem to forget all the lessons we learned before. We them re-learn them again, much more quickly and just as painfully.

The same thing happened with crypto. Crypto started with some correct assumptions like "online services that stop working when it's a holiday are ridiculous", "it should be possible to send money to a friend without your government knowing about it", or "money transfers cost nothing to execute, so they should cost nothing." It then promptly threw out the entire framework of banking regulations, quickly re-learning why most of those regulations existed in the first place.

As I read this, I'm also working through a pretty dense feature that took a fair bit of iteration. The end result is actually significantly less code than it was about halfway through. And I was wondering if the AI actually helped me at all, since surely I could have written the code in the same time it took to iterate

But! Because of AI I was able to rapidly hack out like 4 variants of this feature that I didn't like. And felt comfortable throwing them away just as quick.

The main insight here I think is that LLMs are great tools for iterative development and iterative problem solving in general.

You can very effectivly iterate alone using the LLM as a mirror, rephrasing what you put in and adding a bit.

You can use LLMs to quickly create prototypes to give to other human beings to help you with the next iteration.

If you get something from someone else to iterate on you can use the LLM to help you with understanding to rephrase things in a way more suitable for your understanding.

But instead everything anybody seems to be talking about seems to be one shoting things and AI iterating with other AI.

The big problem here is that the one thing AI does not have is agency. The naming AI agent is wishful thinking and marketing.

I find that it really is effective when you iterate and plan and review, but the problem is more psychological on the human side. It's just too easily available to take the lazy option and just let it do the thing, postpone the thorough reviewing and you end up in a similar situation as tech debt. In an ideal world with no deadline pressure and infinite discipline, AI can be used in productive ways for sure. But when you actually write the code, there is more of a "do you do it or not" switch, and with AI it's a smooth ramp, you can be just a bit less involved or just a bit more. And I end up feeling like I'm not fully involved, I'm halfway working and my whole mind isn't tuned into it properly. I'm not sure how to express it. Also, now several months in, I just don't get the same feeling of accomplishment from the little wins. It's too automatic, doesn't feel earned.
The anchoring thing is what gets me. Once I've seen the AI's first try, even when it's wrong, I can't really write fresh in my head anymore. I end up editing instead of starting over. Code quality usually ends up fine. Time-wise it's a wash or worse, you just don't feel it until you look at the clock at end of day.
I think AI exists to make humans better, not to replace us (which it can’t anyway). I use LLM’s with new topics answer questions and tutor me (for instance with multivariable calculus -course this spring I asked Claude to create 10 practice exercises, which I then did and it reviewed. Harder ones it did with me step by step.) hopefully not needing them after awhile, when I gain proficinency. Automating humans away is not going to work. There’s a reason why we are the apex predator and ruled this planet for million years.
So I am figuring out how to let LLM write code automatically as long as I clarify the requirements. I have made a set of skills to deal with this and it called tdd-pipeline. I eat this dog food and by several rounds of iterations to fix bugs, it works better and better. Now I feel much relax while it is working.

I open sourced it on GitHub, you may search alexwwang/tdd-pipeline to find it if you are interested in it.

On the other hand, some companies are pushing the idea that engineers should build robust self-evaluating agent pipeline with human feedback in the loop so that agents write most of the production code. Creao's CEO said that they rearchitected their entire production systems in two weeks this January. He also claimed that their agents implemented so many features so fast that they had to wait their business development to catch up.

I wonder how we can evaluate these two options: using AI to 100X the output versus using AI to advance one's craft.

In the meantime, the productivity gain of AI is real. Case in point, An engineering org of Snowflake has met all its OKRs ahead of time in the first quarter for the time in the company's history. It had never happened, and usually meeting 70% of the planned OKR would be considered an achievement. I can imagine the stress of the engineers when they see such outcome.

This approach works well, and leads to better code being shipped. The key disconnect for me is not always the code being of high quality, but ensuring my understanding of the "Why" of the bug and the fix is good enough to justify it and then also explain it if the time comes.

That said, I'm learning to let go as much as I can and trust these things when it's "safe" and seeing how that shakes out. The risk is something falls over and I don't know how to fix it (of course) but I know it's a risk and I'm trying to avoid it so it probably won't be as bad as I catastrophize.

This sounds very right. My rule of thumb is at least 50% of the AI work is supervising, reviewing, guiding and bug-finding. A codex session that started a pretty significant chunk of work this morning has just informed me that it thinks that it's done,

Final usage: 5,245,839 tokens over about 10 hours 3 minutes.

It's quite remarkable that the result looks actually correct. But through all this time two more agents (one claude code and another codex under pi) were working alongside reviewing, criticising code, and finding bugs. It would have been impossible otherwise.

I do something similar. Design reviews are an extremely valuable part of the process but they take a long time and people are busy and who really has enough expertise to provide meaningful feedback? Well, the AI is always available and it can rip through my entire codebase in just a few minutes and it has seen more code than I could in a thousand lifetimes. Asking it to review my ideas, my proposals, my implementation, my documentation - anything I can think of - generates a wealth of good insights.

That's not to say that I follow every suggestion it gives any more than I accept every line of code it generates. I'm still the engineer and I still have my reasons for some decisions that may not make sense to an outsider. But as a rubber duck or a second set of eyes or a sober second thought, it can offer a lot of good feedback that I can consider, and most of it is quite good.

I'd say about 8/10 suggestions it makes I agree with and accept, maybe with some small tweaks. I don't think I've ever seen it suggest anything stupid. The last 2/10 are generally either things that would make sense if not for the rest of the context in my head that says it is that way for a reason, or because it's too minor to worry about changing.

All of this extra review takes more time and it usually results in me making a bunch of extra changes I wouldn't have otherwise made, but that's all I'm the make of much better quality and reliability and that pays off in multiples in the future.

I use cheaper models (Deepseek is king, but GLM and Kimi as well) and do the planning myself. I often start a task myself, write some code to get the LLM on the right track, and then have it complete parts of the implementation that are kind of boring or repetitive. LLM's are just next token predictors, I don't mean that in a demeaning way, but I've found if I can get the LLM started on the right track with my own code, it completes what I want. Having the LLM write code just from a spec ends up with poor quality slop in my experience.

I'm not 100x'ing my output like some people claim, but using it as a augmentation rather than delegating my work to it results in better code, and I don't lose context / control over my codebases. I really have read 100% of the code, because the LLM is generating smaller pieces around and inside my own written code. Works well enough for me, and open models are already both cheap enough and good enough for this workflow. This is why the big companies are so desperate to push full-on agentic hands-off workflows and developer replacement - that's the only way they won't go bankrupt.

I think using speed to describe the rate of progress in software development is where the frustration comes from. Software isn't a velocity thing. It's a space thing. It's memory. Information in some media. You can transfer a billion bits in less than a second. The time domain is largely irrelevant in business terms.

Having taste and the ability to author high quality prompts is still the most important thing. It was always the most important thing if you think abstractly about how all of this works.

Optimizing for code quality over raw output speed is a great approach. The time 'lost' writing it slowly is easily made up by the time saved on debugging and maintenance later.
My workflow now

1) While walking voice chat with ChatGPT about architecture and various interesting angles for a feature or product

2) Have it create summary of things we talked about

3) use that to seed spec development phase

4) write comprehensive specs using both Claude Code and Codex

5) create todos from specs

6) implement todos using both Claude Code and Codex to check each others work

7) run focused code check prompts e.g. specifically for error handaling, concurrency issues etc. They tend to find more issues in these focused passes.

This is exactly the reason why I like to work with local models on a regular specced machine. The fact that the agent moves slower allows me to stay in the loop much better, compared to skimming through a huge amount of generated content and data and then going to the end really fast to make sense of it all, in the interest of time (and thus losing track and quality). The fact that I can run it locally makes it (much) cheaper too.
I think my current conclusion is that AI makes <foo> more important than ever.

I’m not exactly sure what <foo> is but I feel it. I think it’s quality and authenticity and craftsmanship. That difference between an expensive tool and a cheap one that you can’t easily describe but you just know it.

Is there a word for this? I bet the Japanese or Germans have a word for this.

I use AI a lot now. But I also do it in small steps. It isn’t a craftsman, but it can help me be one.

Exactly. That is what we do. We do software that can kill people and it is very sophisticated, like controlling robots and we prototype using LLMs and it is amazing.

People believe that you can only use LLMs for sloppy programming. But you can also use it for writing ten times more code of Swiss cheese model tests, and domain specific languages.

You write ten times more code than necessary and all that extra code is testing. Projects like SqlLite do that because they need to be perfect.

Before LLMs we had to use engineers for that and it was a painful and repetitive work, and they were always late and made much more mistakes than LLMs, specially because it was dull and tedious for great engineers to spend their time into.

Now we write tests and when all test pass we write new test for checking the tests.

We divide each complex problem in small subproblems and we warrantee each of them by formal means. We have multiple ways of solving the same problem, usually with one brute force solution that is simple and warranted to work but inefficient, and we can use it to compare with more efficient methods.

Before machines could do that, people doing that were burned down and exhausted, and always leaved pending work to complete.

I believe this thinking can be abstracted to software design via AI in general. If you are thoroughly prepared, and keep things simple, it's incredible what help Claude or GPT can be.

I have Claude basically doing all the coding for me for a simple game I am making. However I don't consider this vibe coding. I spent several hours thinking out the design on a piece of paper, playtesting it in person. I came up with a list of potential mechanical issues within the game, and asked Claude to come up with more. It found more issues, and we solved them all together. Once the game was mechanically sound and edgecases were solved for, it built an MVP. I ran the program, and found more bugs. I came up with my own solutions, and Claude did the same, and we figured out which were best to implement. Claude then wrote more code, and raised issues when they came up, and we worked through them together. I'm incredibly happy with how its turned out so far.

I used LLM as a tutor to tackle unfamiliar terrain. That is, I write code that I know very likely doesn't work but is the best code that I could have written. The LLM will happily tirelessly show me what I did wrong and what the correct code actually look like. Then, at the end of it, I got code that running. That's a tight feedback loop.

It's still very slow. It took me two hours to write code that generate JSON data and then to write a web page that displays a knowledge graph.

One thing you have to be aware is that the LLM will happily generate code for you and you have to discipline it from time to time. I notice that my reading comprehension begins to suffer if I don't write the code myself and have to understand what the LLM wrote for me as opposed to the LLM correcting where I went wrong.

One thing I would like to try with an LLM is understanding a large and complex existing codebase like OpenSCAD that doesn't leverage my existing skillset(high level programming languages with OpenSCAD as primary language in the past year). That has always been a barrier to contribution for me.

Are we overcomplicating AI by approaching top down, so naturally there are trillions of variations and too many ways to fail? Supervising a component-level scope, with emphasis on quality control (regression, perf testing, benchmarking, etc), seems to produce great work.
Hot take, barring from special edge cases, I find using dumber models (like local Qwen 3.6) to be the best balance. Smart enough to do stuff but dumb enough where I don’t trust it and verify what it’s doing rather than letting it do the third whole code base refactoring of the day. Also forces me to know my code base and ask very descriptive tasks rather than go “something is wrong, fix it”.
If you have code chock full of bugs (hand written, or LLM generated?) then I guess any tool that helps find them is good. I find that traditional static analysis tools like Fortify are next to useless because of the low signal to noise ratio - flagging things that aren’t errors or vulnerabilities just because someone decided that certain standard library calls should be flagged etc.

I wouldn’t be praising an LLM for it’s voluminous output unless it is doing a much better job of finding actual bugs, and if the code really is that buggy then it might be worth asking why!

100% agree after building a production ready platform ground up. it took 3-4 months but without AI i would never had been done with a team of 3. one thing to note that AI is weak at Front end. So, we did the entire front end without AI.
This was a really good article. I typically code with Sonnet or Opus, but I'm going to start using Codex to review local changes before I do a PR. It does a great job at identifying issues and then patching them. I just feel weird having Anthropic review it's own code so this will be my workflow for now. For complex features I also use Opus to cook up an implementation plan ahead of time that a lot of the time I have Sonnet implement. I may also have Codex start devil's advocating the plan before I start too, just to catch issues earlier.
The bottleneck moved. It used to be writing code. Now it's knowing what to ask for, in what order, and how to validate what comes back. That's a fundamentally different skill than coding.
I'm pretty intensely disinterested in "agentic" coding. However, this was very much the inspiration behind the custom Claude-backed Goose agent I deployed into some of our Gitea repositories over Christmas--the less sensitive ones, of course, I'm not sending our proprietary code to Dario.

It does short and sweet code reviews, and going back and forth with it is, as often as not, slower than just typing and merging the code.

I'm quite pleased with it as a middle ground.

Yes! That's what I've been doing at work for the last few weeks! And while it doesn't appear to be super fast, I'm already pretty certain that the next round of testing will come back with fewer unexpected issues because together with my agent and the right usage, I was already able to catch stuff that I would have missed otherwise.

Also feels much better than pure vibe-coding (which I still do for personal projects that aren't mission critical for anyone).

https://news.ycombinator.com/item?id=48246232

This reminds me the article above. Now people have diverse ideas on agentic coding. Some suggest human-in-the-loop while others suggest giving a detailed specification and let the agent run freely; some suggest leveraging LLM's high productivity and here we get an opinion that LLM can actually slowly write good code.

It's happy to see opinions that are more practical and variant emerging, turning LLM into literally a tool instead of something to be hated or hyped.

In my own practice, I find LLMs (SOTA ones) good at medium-level tasks, those needed to reason and plan for a while. However, the design taste on architecture is unexpectedly disgusting. Sometimes writing interfaces myself and asking LLMs to fill in implementations, alongside context-completing tools like context7, deepwiki, docs.rs MCPs, etc. and giving a escape hatch (e.g. encouraging it to use the AskUser tool in Claude Code), may be considered my best practice.