Item — HN Reader

Being 10x (or whatever multiplier) faster at programming doesn't mean you're going to be 10x faster in designing a product or any other aspect that goes into making a good product.

Even if you hired an actual programmer, it'd take a massive amount of time to build a Photoshop clone.

Of course, at the end Photoshop is lines of code and it could be output as is, end to end. One problem is that users aren't generally giving very precise design documents which would narrow the way to interpret them into code in precisely one way. Or that a design document at any level of precision, other than code, couldn't be interpreted in multiple ways when it comes to a specific implementation.

LLMs also take a relatively long time to output acceptable code, often taking tens of minutes before giving you a small diff. The larger the codebase, the longer it usually takes to start producing code, even over an hour.

1 comments

> Of course, at the end Photoshop is lines of code and it could be output as is, end to end

And yet it’s not.

> LLMs also take a relatively long time to output acceptable code, often taking tens of minutes before giving you a small diff. The larger the codebase, the longer it usually takes to start producing code, even over an hour.

The problem is that they don’t generate acceptable code, they generate code that needs to be edited to be acceptable. That has always been the slow part of engineering. Waiting an hour for a bugfix even if it cost $75 in tokens would be cheaper than hiring an engineer but only if it worked. And it’s a bit like hiring a snake oil salesman - it passes the sniff test but it’s only when you’re drowning in the fact that your ai now takes 4 hours to fix the same bugs, and it introduces new bugs _and_ you don’t have anyone who can reduce that complexity that you see the reality. For a lot of us, that is immediately clear from first glance at the output of Claude and codex and the likes.

> And yet it’s not.

That was my point as well. That it hasn't been output, even though it could be done by a talented solo developer given enough time, and that current LLMs definitely aren't able to do so.

> The problem is that they don’t generate acceptable code, they generate code that needs to be edited to be acceptable.

You've never had an LLM output a one line bugfix that is correct to the point where you don't have to edit it?

To make things more concrete, here's an example from the creator of Redis on how he utilizes LLMs in programming: https://antirez.com/news/164

> You've never had an LLM output a one line bugfix that is correct to the point where you don't have to edit it?

I have. I’ve also had IDEs and static analysers do the same thing. I can also take my car out of gear and have it roll down a hill but that doesn’t mean it can run without fuel. Only a sith deals in absolutes, and in the general cases LLMs don’t generate acceptable code.

My experience is that when I ask for a clear and well defined problem, of the scale of "add motion blur (linear, spin, and zoom) to the filters menu; include standard dialog box (see existing design) for user input on all options", this works something like 90% of the time, is obvious garbage 5% of the time (in my experience, when it claimed to be writing "unit tests" it was actually performing regexes on the source code), and is subtly wrong the other 5% of the time.

If you use the planning mode, and your first move in the project is "write plan to reimplement photoshop" then you blindly say "continue" until the plan is done, then you get 0.9^{number of features} success, which of course on the scale of photoshop is going to be a failure. But this is still in one sense a 10x speedup in that 9 times out of 10 you're only doing code review, not having to re-write it. But code review is a real thing, so it's 10x on writing code not 10x on delivery.

Try it out, like seriously, learn to use it well. Spent a few weeks with it. You will not say these things if you were an experienced user of these tools. Saying their code is "unacceptable" is a skill issue. Describe what you consider "acceptable code" and watch it produce it in copious amounts. They don't have one mode, one setting, they can generate whatever the F you want in whatever style you deem "acceptable". You're completely in control.

That said, I watched many of my - generally pretty clever - colleagues struggle mightily with this. I can't put my finger on it yet. Regular "programming" - typing BS syntax one character at a time - always felt astronomically boring to me so I'm one of the guys happy with these tools. Not happy with how it will fuck up society though, but that's uh.. yeah.

As always in these discussions, I think people compare apples and oranges. LLMs are great with in-distribution solutions for solved problems with a lot of relevant prior material and in established technologies. Frontend stuff works great, for instance.

But for novel solutions, complex business logic, things deeply integrated with external systems... the code generation quickly turns terrible and useless. Especially if it's in anything but Python, Java, or JS.

Most of these differences in results end up being about differences in application. LLMs suck at out-of-distribution material, inherently.