“Vibe Coding” vs. Reality

I think the bigger "AI hype vs. Reality" gap is about the productivity numbers people casually throw around, like "10x as productive" or even 100x.

For example, here are YC partners quoting a company in a batch claiming "100x speedup" in coding performance compared to the previous month:

https://www.youtube.com/watch?v=IACHfKmZMr8&t=1837s

You can tell this claim is false, because that level of productivity increase would be glaringly obvious to an outside observer; it wouldn't need to be self-reported.

A YC summer batch is 84 days culminating in Demo Day. So a 100x speed improvement would be like a team spending less than 1 day of coding and ending up with something that's on par with Demo Day in terms of functionality. Maybe the design would be wrong, but that wrong design would be just as fully-featured as a Demo Day app.

So if 100x were true, the partners in that video would be talking about how the new batch dynamic is "They get breakfast with a customer, learn something new, have an epiphany, and then later the same day they have their entire app rewritten based on what they learned, and that scratch-rewrite is already at a Demo Day level of functionality." The partners aren't talking about that dynamic because it's not happening. So clearly the self-reported 100x is inaccurate.

Even 10x would result in partners saying "Whoa, in this batch people have a Demo Day-quality app in production by the end of week 1 instead of week 12." The partners have a huge sample size on how much teams get done in what time period, so it would be glaringly obvious to them if this batch were shipping 10x as fast as previous batches.

That external observation would be the headline if it were what the partners were actually seeing. Since that's not the headline, it's clearly not what they're seeing, so 10x can't be the number either.

At this point in time, we're following the time corporate got on the outsourcing craze step for step. It had all the hype, hands off, cheaper for the same work, faster to market, every other argument you've all certainly heard. Then the reality hit. The whole discussion around LLM coding agents feels indistinguishable.

This is the go community saying a computer will never best human go players.

We already have examples of a model finding more performant sorts [0], given the right incentives and time, and the right system for optimizing (LLMs trained on “average code” probably aren’t it) the computer will best us at creating things for the computer.

Is “vibe coding” real today? Not in my experience, with even Claude code. My hand has to be firmly on the tiller, using my experience and skill to correct its mistakes and guide it. But I can see the current trajectory of improvement, and I’m sure it’ll get there.

[0] https://deepmind.google/discover/blog/alphadev-discovers-fas...

I'll share a new wrinkle that casts more shade on the coding LLMs.

We have a fair number of offshore resources that are used for dev. They developers are fully integrated into the team, are in all the stand-ups, and substitute for the usual role of junior programmers. They don't get the grunt-work shoveled on them, they get the same work as everyone else, they're just expected to not be as fast.

In 6 months 2 out of 4 of them been sacked, and surprise, not because we could replace their work with LLM output, but because their use of LLMs was so unrestrained and scattershot the pull requests they submitted had become nightmares. One thing mentioned in the article about unit test creation was something we saw as well. Perhaps this is partly due to working an existing code base where the LLM loses some of its advantage, and certainly some of it was cultural in that progress was thought more important than actual manageable code. The two sacked fellows where told, literally, from my own mouth, multiple times, "You cannot just ask Copilot to write you code, paste the entire thing into Visual Studio with no thought of what has changed, with the end goal of just compiling and meeting the single set of acceptance criteria on your story. You're breaking other things and introducing bugs." It went on deaf ears, and now they're gone. They were nice people, I didn't know how to get through to them, but they were convinced the LLMs were the way to go.

I use LLMs to help write code every day, and I wouldn't want to be without it, but I'm fairly surgical about it. Most of the time Copilot gives you a page of say, React code, or EF Core queries, you have to be really careful about anything you didn't explicitly ask for. Honestly, there is a time savings, but there is not a quality increase. The benefit is subverted by the time it takes to figure out how to ask correctly, the time to vet the output, and the time to fix the little tiny insidious bugs it can introduce.

So, don't go vibe coding and lose your job, is something to think about. I have to admit that it has worn me down meeting these interesting people from far-flung locations only to watch them flounder and get let go.

> "Vibe Coding" might get you 80% the way to a functioning concept. But to produce something reliable, secure, and worth spending money on, you’ll need experienced humans to do the hard work not possible with today’s models.

This would have been clear from Karpathy's full statement:

> It's not too bad for throwaway weekend projects, but still quite amusing.

I hate how what is effectively a stupid meme phrase became an actual term in a few days.

> "Vibe Coding" might get you 80% the way to a functioning concept. But to produce something reliable, secure, and worth spending money on, you’ll need experienced humans to do the hard work not possible with today’s models.

The problem is that 80% of the job is a proof of concept at best. 80% is effectively a QA walking into a bar[1].

[1] https://barrypopik.com/blog/a_software_tester_walks

Over the last week I tried to use a combination of Claude and OpenAI o3-mini to do a direct conversion of about 500 lines of uncommented academic modeling code from Matlab to Python. I can’t stress enough how badly these models performed. Nearly every consequential line had some variety of off by one or logic error, often very subtle. I didn’t try cursor or the more agentic systems, but I would be astounded if they properly rigged up a test harness, inspected the output and were able to respond to the runtime errors. I’d be happy to share the code if anyone wants to surprise me.

This is exactly the kind of semi-mechanical, low added value work that would greatly benefit from automation, and they really fell on their faces. I really benefit from these models on greenfield tasks where I can delegate minor drudge work, but in this case I honestly think they actually increased the difficulty.

I've been "Vibe-TDDing" all afternoon and I'll tell you what, vibe tests are better than no tests.

And so long as you have some decent-to-solid understanding of coding and testing (this is non-trivial, I've been coding for professionally for ~20 years) then you can direct the machine to put up decent guardrails first, and then you can kinda go nuts and let shit grow, prune it back, repeat.

Basically, if you know what code/tests ought to look and act like, then you can significantly reduce the negative externalities of having LLMs do your coding for you.

> ever since I started to share how I built my SaaS using Cursor > random thing are happening, maxed out usage on api keys, people bypassing the subscription, creating random shit on db

This has to be a troll no?

This idea that now everyone with little knowledge can code is absurd. For sure everyone can code: with hours of dedication, sitting down and trying things out, learning and improving from errors and past experiences. There is no other way round. I don't know how the next generation of coders is going to be like, but my advice still stands: read books, realiable sources, do your homework and "vibe coders" will become so irrelevant that will be extinguished by their own ignorance. Don't get fooled by number crunching programs that seem to "program".

It makes me feel very secure in my job that so many engineers ITT are downplaying the ability and productivity of AI coding tools. You can pry cursor out of my cold dead hands. If you aren't seeing a 10x boost, then you must not have tried it lately, or haven't got the experience to prompt well.

What it excels at: - Boilerplate code that's been written 1000x, which can saps your time and enthusiasm for the meaty problems beyond that.

- Complex DSA work. It has been demonstrated millions of times in training material.

- Simple and tedious tasks like making dummy data for tests and struct literals.

- Tightly scoped refactors.

Where does it falter?

- Mapping your product/business to the code or abstractions needed. I think this is where junior devs struggle to leverage it.

- Doing large scale multi-file refactors without proper specifics, guidance, and context. It also can't write a huge project from scratch. Humans are still need to fit the pieces all together or provide guidance. I think this gap closes soon.

Code quality simply isn't a problem IME. If it didn't one-shot your dream abstraction, you probably weren't specific enough in the prompt. Most human-written code is also junk, so pointing out a minor gaffes isn't really a dunk on AI. It's still a massive productivity booster if wielded by even a half-competent engineer.

Hot take: Vibe coding is going to be the new Excel of technical debt.

Most tech savvy places will avoid it, most good programmers will never encounter it. A bunch or us will make a career out of fixing the mess it makes after it explodes.

My first real job was doing just that at a broker trader which lost 10m on a trade made by an Excel spreadsheet that used a stale yahoo finance API to get exchange rates.

I have to say I am extremely sick of this term.

Vibe coding is my favourite fad so far. Much like the last few fads I’m going to make so much money out of cleaning up afterwards it’s unreal.

Really this whole industry is on another fucking planet. I hate it but it’s so easy to make money.

The biggest give-away is: When you look at the companies making these claims and their jobs page shows they are actively hiring developers.

This brilliant piece of satire from Steve Yegge got buried for some reason:

https://news.ycombinator.com/item?id=43446695

Judging by the comments, most people couldn't even tell it was satire, which goes to show how absurd the hype is right now (and probably why it was buried).

I finally gave in and clicked on the article so I could find out what "vibe coding" is.

You learn something new every day. Some days that thing does not piss you off. Today is not that day.

ThePrimeagen is doing some kind of vibe coding ad on twitch right now trying to build game in 7 days. There are 10x coders in the room and two days later they are struggling with hilarious basics like off by 1 errors while tweaking something that could be described as donkey.bas.

    "ever since I started to share how I built my SaaS using Cursor"

    random thing are happening, 
    maxed out usage on api keys, 
    people bypassing the subscription,
    creating random shit on db

    as you know
    I'm not technical so
    this is taking me longer that usual
    to figure out

- (leo, 2025)

New silver bullets are the same as old silver bullets, but these are easy to fire at your own feet.

The more I work with LLMs and try things like "vibe coding" the less worried I am about AI taking my job any time soon.

In the right contexts, I find LLMs can speed up my work a lot. But it's nowhere close to being able to replace what I do.

> Cursor has some sort of "concise mode" (archived) that they'll turn on when there is high load where the model will still be rated at the normal price but behaves in a useless manner. This mode will omit details, drop important findings, and corrupt the output that is being produced.

This is a real problem that I have experienced on and off. It's getting to the point where everyone on my team is actively looking for alternatives. Generally, I've found Cursor works correctly after business hours. But, it's increasingly giving absolutely useless responses during business hours.

-----

That being said, I agree with many of the author's observations. However, for me, it's not really a breaker. It's not much different than working with an intern or junior engineer. If you ask them to do too much all at once, they come up with bad solutions. Plus, they have a tendency to make "dumb" decisions.

For me, I've found solutions for nearly all of the listed issue. Much of it comes down to being diligent during code review (like you should). For example, the Typescript issue, I come back later to have it fix it.

Specs are the one that still baffles me. It's absolutely terrible at writing proper specs. In particular, it falls into a really bad cycle whenever there are errors. I don't have a solution for this one.

The reference to Claude Plays Pokemon isn't applicable to the discussion of vibe coding, although the suggestion that AI agents can fix the issues with vibe coding is funny in an ironic way given the disproportionate hype around both.

The issues with Claude Plays Pokemon (an overview here: https://arstechnica.com/ai/2025/03/why-anthropics-claude-sti... ) is essentially due to the 200k context window being finite, which is why it has to use an intermediate notepad. In the case of coding assistants like Cursor, the "notepad" is self-documenting with the code itself, sometimes literally with excessive code comments. The functional constraints of code are also more defined both implicitly and optionally explicitly: For Pokemon Red, the 90's game design doesn't often give instructions on where to go for the next objective, which is why the run is effectively over after getting Lt. Surge's badge as the game becomes very nonlinear.

Although, both vibe coding and Claude Plays Pokemon both rely on significant amounts of optimism around the capabilities around LLMs.

My, very limited experience with LLM assisted coding is that it depends... For basic frameworks done in something like Python it is very good, but not perfect, yet. But the iteration cycle to get to where you want to be is still faster than doing the whole job manually and I see this as a big win.

For more esoteric fast changing languages/frameworks it has me chasing my tail in a chain of code updates where each fix breaks something in the n-1th, or n-2th version. Sometimes it's deprecated code, or it halucinates functions that would be valid if your were using a a different language of framework. And sometimes simple coding errors.

But it will get better, a lot better.

The main benefit is that it will let a invested non programmer client build a functional framework prototype and then combine that with a list missing features that a more skilled programmer can flesh out to a first cut solution.

For the first time we 'might' get better requirements with an actual working model instead of having the implementor doing most of the requirements as a first pass from a high level hand wavy requirement. I think we're going to see some amazing tools for this.

What I don't see it doing is creating original algorithms to solve things being done for the first time.

I've been messing with this for a few days now so I'm not going to claim to be any sort of expert but as someone who has been coding for more than 20 years I do appreciate the set it and forget it nature of being able to throw q developer or whatever at a relatively simple problem that I'm curious about and let it crank away for half an hour while i'm working on something else. I've tried it on a couple of reasonably small and well defined problems, mainly focusing on python, and it works surprisingly well. It'll run the scripts and fix errors and can suggest prompt improvements. I've also tried it in a large codebase with much less success, so YMMV.

Also it is important to be able to review the code because it could be the case that it looks mostly correct but has some subtle errors in it that can mislead you. For example I was trying a couple of different ways of computing some indices that have a bunch of variables and one way had a mask that made no sense involved. "Vibe coding" without being able to check the work of an LLM is almost certain to go poorly, IOW.

I think it's a good thing. Let them experience our little hell. They'll come out appreciating our craft more once they get through :)

Making the arguing that these tools have flaws seem like a losing battle. Soon those flaws will be fixed[1] and youll have to find new flaws to complain about. Eventually hopefully you'll realize that you just don't like feeling displaced.

[1] it's unbelievable what a difference in quality 1 year made for chat gpt

> For now, they are worth evaluating and discussing, but are not ready for us to delegate the precise task of creating reliable, secure, and scalable software that powers our society.

The good thing about vive coding is it avoids the software development lifecycle completely from the user perspective in a platform that has an integrated SDLC which means from defining idea to ensure visibility in changes to a runtime where the user can see it. In my mind, modifying without a hassle in a controlled environment is what users look for. Software development assisted by AI will be a thing for engineers but Vive coding is aimed for users outside of engineering. I sadly see only a handful of companies being able to pull this off.

"Vibe Coding" might get you 80% the way to a functioning concept."

But 80% of the functionality is only 10% of the work. The last 20% of the functionality remains and will require 90% of the work.

100x would mean after one month, people'd say 'I just did 10 years of work.' Is anyone saying that yet?

If you believe in vibe coding, surely you are holding a massive short position in every major software company, no? I mean, surely, any day now, a bored student will vibe code a full replacement for a major software package and destroy the income of the SW giants one by one, right?

Wake me up when someone vibe codes a Chrome replacement, or an iOS replacement, or MS Office...

Except we know this won't happen anytime soon because we all know vibe coding isn't very useful beyond toy projects that leverage complex libraries written by actual developers.

I mean ... most code out there is pretty bad, so LLM assistants contributing pretty bad code just keeps the mean where it is. And obviously it has to be, how can anybody expect an LLM to produce output with quality that's higher than its training input? Expecting that is appealing to magic or some consciousness that doesn't actually exist or just plain anthropomorphising.

If you are working at a place where that quality level is standard -- and let's face it, a large number of companies produce average or below-average quality code (by definition) -- then using an LLM assistant isn't that bad. At least if such an assistant doesn't have some extra flaws beyond producing the best summary of its training data, which is exactly what an LLM does. It actually justifiably replaces developers in such an average-or-below place. But if you are aiming for the top end of the quality scale then there is no way this can be achieved by LLM output. Purely on principle.

This shouldn't even be a controversial opinion. I'm quite surprised every time this is questioned or even just debated.

Wait... is this really a thing? Do people think that they can actually have an LLM write code for anything but a simple program for them?

This should be called "vape coding".

Strongly agree with the article, and happy to see so many lucid people, comments and articles on HN that thoroughly deconstruct the "vibe coding" illusion.

Also, Andrej Karpathy really disappointed pushing such brittle BS as a revolution.

Not all AI-assisted programming is vibe coding (but vibe coding rocks) - https://simonwillison.net/2025/Mar/19/vibe-coding/

I wrote this because I was worried that "vibe coding" was being misinterpreted to mean "any time an LLM outputs code", as opposed to the intended definition of code where you deliberately don't review the code and see how far you can get.

Whether vibe coding works for you depends on how tight your requirements are.

Is it just me, or has anyone else noticed that:

1) Cursor has been crashing several times an hour for me recently.

2) Cursor seems to ignore .cursorrules files. I'm using the json format that's supposed to let you filter on file name patterns (although how that works for cross-cutting agent stuff I don't know).

3) Cursor is obsessed with making sketchy iffy defensive code checking for the most recent symptom and trying to guess and shart its way out of it instead of addressing the real problem? And it's extremely hard to talk it out of doing that, I have to keep reminding it and admonishing it to cut it the fuck out, fail instead of mitigate, address the root cause not the symptoms, and stop trying to close the barn door after all the horses have escaped. It's as of it was only trained on Stack Overflow and PHP manual page discussions.

Saw this submission earlier in the day and chuckled. The whole "vibe coding" thing is hilarious, and I say this as someone who heavily leverages AI in my coding tasks.

Moments ago just saw this ad at the front of HN-

https://www.ycombinator.com/companies/domu-technology-inc/jo...

I honestly am not sure if this ad is a joke. I assume not, which is hilarious. Put in you 12-16 hour days for hilariously bad pay, and your onboarding will be doing one of the most pathetic, deadbeat jobs possible which is making collection calls. And your "vibe coding" is to use voice agents to...make collection calls.

Must be pretty grim pickings if this trash is getting advertised on here.

try writing next.js 15 using AI, and you understand how far we are away from getting replaced by it.