▲ 276 points
back
82 comments
One consideration not mentioned is around developer sophistication. Steve alludes to the expansion effect of CodeGen ("there are millions and maybe billions who are jumping at the chance to code"), but doesn't consider that the vast majority of these people don't know about arrays, data structures, memory, containers, runtimes, etc, etc...
To me, that's the most important consideration here. Are you targeting professional devs who are enhancing their current workflows iteratively with these improvements? Or re-thinking from the ground up, obfuscating most of what we've learned to date?
Maybe we need to trudge through all of these weeds until software creation hits its final, elegant form where "Anyone Can Code".
Maybe the old Gusteau quote is actually fitting here:
"You must be imaginative, strong-hearted. You must try things that may not work, and you must not let anyone define your limits because of where you come from. Your only limit is your soul. What I say is true - anyone can ̶c̶o̶o̶k̶ code... but only the fearless can be great."
Before I finished the quote I was like, “…the ratatouille guy?” Haha, great quote though.
Anton Ego: "Not everyone can cook, but great cook can come from anywhere"
Well we'll never reach a state where anyone can code. I have pans, a supermarket nearby, cookbooks and a belly, still I'm never gonna be able to cook, I snooze after 30 minutes, even if I succeed once, I get bored and stop for months etc.
Simplifying to the point a grandma could make an app isn't gonna make any grandma WANT to make apps. And that's fine, there's no issue, we don't have to make more people code and those who want, will, even if all we had was assembly and a light board...
Which I think is the spirit of your quote basically.
That’s a bad comparison, cooking has been done by people for thousands of years, your problem with cooking is laziness, there is nothing mentally or physically stopping your from learning to cook.
I do agree with your second paragraph and it’s more that you DON’T want to cook versus you being unable to cook.
Windsurf + Haskell w/ CLI tools has been pretty amazing. Windsurf's agent will loop for minutes on its own to figure out the right structure of a program. You just need to tell it to:
- use the hoogle cli to search for the right types and functions
- include a comprehensive test suite
- run a build after every code change
- run tests after every successful build
GHC + a Claude-based agent is a thing to behold.
Woah, that sounds awesome! I'd love to see how you set that up and how much it can do without your intervention/approval for various actions. Might you have a video of your workflow that you could share?
Sure, I'll record a little something with Loom for you tomorrow.
What's the maximum file size for which this is useful in your experience? I have been refactoring some project solely to enable AI code editors to edit it. Some users in the discord suggest a maximum file size of 500LOC or small, which seems unreasonable.
> The next big thing was Cursor. I must admit that I never personally fell in love with it, but given how many people I respect love it, I think that’s a me-problem
I've met so many engineers who have said exactly this. There are clearly some group of people obsessed with Cursor, but it's interesting to me how alien they seem to the majority of people using ai codegen right now.
i had to uninstall it because it had associated itself with every possible file extension. i couldn't open a file without cursor popping up. very horrifying for that to happen to my computer when working on important projects
I uninstalled when I found out it overwrote "code" command in WSL2 Ubuntu without my consent.
"code" used to fire VS Code.
My rationale is: what else do they think they can get away with in my system?
VS Code Copilot Chat with #codebase in prompt has the edit mode which behaves similar to Cursor. Even more so with o1-preview selected.
On Linux, I have the opposite issue. I ended up hard symlinking cursor to VS Code because Cursor wasn't opening despite being set as the default editor.
I use it. It’s not a revolution, but it’s an upgrade over vscode+copilot. Only a matter of time until the two have feature parity, though.
I prefer vscode+copilot. It's much cheaper and has all the functionality I want. There's access to 3.5 sonnet, and it can edit/create up to 10 files at a time.
I'm pretty much full time on cursor from vscode. I don't trust it for big code blocks, but a control-k + reasonable command (I could have typed up myself) is saving me quite a bit of time.
This post is the latest in a series about Townie, our AI assistant.
Our first had a nice discussion on HN: https://blog.val.town/blog/codegen/
The other posts in the series:
This blog article is written in a very engaging way. It seems to be more or less a masterclass on how to keep someone's attention, although there is no meta-story making you wait for the big fulfillment at the end.
I think it is the short, punchy sections with plenty of visuals and the fact that you are telling a story the whole way through, which has a natural flow, each experiment you describe, leading to the next.
Interesting. On lmsys, Gemini is #1 for coding tasks. How does that compare?
It's really interesting seeing the progression here, integrating AI-assisted coding tools into something like Val Town is a great arena for exploring different patterns for this stuff.
Worth checking out their Cerebras-powered demo too - LLMs at 2000 tokens/second make applying proposed changes absurdly interactive: https://cerebrascoder.com/
I wonder if you didn't try cursor's Composer tab, especially set to Agent?
I didn't care that much for cursor when I was just using Chat but once I switched to Composer I was very happy, and my experience is in total disagreement that it's not so good for smaller projects.
They also must have a good prompt for diff-based completions, I don't know how hard it is to extract that.
Ok, good to know! I'll have to find time to try out Composer
Yes, I wonder if reilly3000 will swing by with a leaked system prompt from them too
> The biggest problem with all current codegen systems is the speed of generation
I don't see this complained about nearly as much as I'd expect. Groq has been out for over a year, I'm surprised OpenAI not acquired them and figured out how to 10x to 20x their speed on gpt4.
Yeah I don't agree. I'm building a product in the space, and the number one problem is correctness, not latency.
People are very happy to sit there for minutes if the correctness is high and the quality is high. It's still 100x or 1000x faster than finding 3rd party developers to work for you.
I wish the models were getting better but recently they've felt very stuck and this is it, so agent architectures will be the answer in the short term. That's what's working for us at srcbook rn.
I'm interested in what stopped you from finishing diffs and diff based editing. I built an AI software engineering assistant at my last company and we got decent results with Aider's method (and prompts, and hidden conversation starter etc). I did have to have a fallback to raw output, and a way to ask it to try again. But for the most part it worked well and unlocked editing large files (and quickly).
Excellent question! We just didn't have the resources at the time on our small team to invest in getting it to be good enough to be default on. We had to move on to other more core platform features.
Though I'm really eager to get back to it. When using Windsurf last week, I was impressed by their diffs on Sonnet. Seems like they work well. I would love to view their system prompt!
I hope that when we have time to resume work on this (maybe in Feb) that we'll be able to get it done. But then again, maybe just patience (and more fast-following) is the right strategy, given how fast things are moving...
I am very biased on this topic.
I have started using AI coding assistant and I am not looking back.
This comes from an engineer that KEEP telling the junior on his team to NOT use GenAI.
The reality is that those tools are POWER TOOLS best used by engineers very well versed in the domain and in coding itself.
For them, it is really a huge time saving. The work is more like approving PR for a quite competent engineer than writing the PR myself.
My tool of choice is Cline, that is great, but not perfect.
And the quality is 100% correlated to:
1. The model
2. The context window
3. How well I prompt it.
In reverse order of importance.
Even an ok model, well prompted gives you a satisfactory code.
> The reality is that those tools are POWER TOOLS best used by engineers very well versed in the domain and in coding itself.
I'm starting to get a feeling of dread that our entire engineering organization is digging itself into a hole with lots of buggy code being written which no one seems to understand, presumably written with heavy LLM assistance. Our team seems to be failing to deliver more, and quality has seemingly worsened, despite leaning in to these tools.
Reading hacker news gives me the idea that LLMs are a miracle panacea, a true silver bullet. I think that the positive stories I hear on hacker news goes through a big selection bias. It has always been the motivated people who always utilized their tools to their best ability.
I definitely don't consider myself to be good in this regard either and struggle to use LLM tools effectively. Most of the time I would be happy with myself if I could just have a solid mental understanding of what the codebase is doing, never mind be a 10x AI enhanced developer.
Matches my observations, having used Github Copilot for several months. Its POWER TOOL.
Val town has been a huge inspiration for the tscircuit site which is basically a typescript playground for electronic design. We extensively use codemirror-ts which was created by Val town and enables typescript autocomplete inside a codemirror editor. I didn’t know about codemirror-codeium but ill definitely look at integrating that as well!
It’s absolutely true that we are in a race for online editors, I feel fatigued competing for ai features instead of building core product features, but since my framework is new, it’s not known by any major LLM providers, so our users can’t get ai assistance unless we build something ourselves.
@stevekrouse huge shout out for your team’s open source work, hoping to help contribute upstream at some point!!
Thanks!! Makes us so happy to see awesome projects like yours building on our stuff
I really enjoyed Steve’s guest appearance on Dan Shipper’s AI & I https://open.spotify.com/episode/6hoiweWkAv84zYTC8DAi5B?si=C...
Thanks!!
> For starters, we could feed back screenshots of the generated website back to the LLM. But soon you’d want to give the LLM access to a full web browser so it can itself poke around the app, like a human would, to see what features work and which ones don’t.
We've had some success[1] with the screenshot to actions - using Gemini/Molmo and ADB on phones. And human like decisions was made by GPT 4o. It also recalibrates itself and says "oh we are still at the home screen, let's find the gmail app first"
1. https://github.com/BandarLabs/clickclickclick - Letting AI control/use my phone.
Kinda got the the same conclusion than OP building in the same space. There is so much innovation going on currently that whatever you do today, two other people will do better tomorrow. Which is a good news for us but difficult time for builders.
Ditto. This was my conclusion after spending a bit of time building https://robocoder.app/
Coupled with the fact that devs prefer open source tools and are capable of (and often prefer) making their own tooling, it never seemed like a great market to go after. I also encountered a lot of hostility trying to share yet another AI developer tool.
(Note I am one of those developers who prefer open source tools — which should’ve been a hint…)
goodnews is the market is still too early, a lot of people still dont know these things exist. as long as you keep showing up you're gonna get a piece of the pie
When can AI do refactors that encompass multiple files or an entire project?
it can do it already, the trick is to prompt it to approach it how a human would.
1. use a temp file as a reference for the entire refactor
2. make it plan the entire thing, tell it to use a high level and low level checklists, tell it to take notes for itself, and tell it to use the temp file as a scratchpad for taking notes and storing code blocks.
3. tell it to do small incremental changes, and do bottoms up approach.
It can already, try cursor composer or copilot edit sessions.
I find that even CoPilot can do this pretty quickly if you do one example refactor and then prompt it to repeat the example on all the files from a find and search.
Since that's already a huge speed up, I'm sure many of these agents can do the same.
And now Claude can do it with the right MCPs installed + smart prompts
"Is this fast-following competitive or is it collaborative? So far it’s been feeling mostly collaborative. The pie is so freaking large — there are millions and maybe billions who are jumping at the chance to code — that we’re all happy to help each other scramble to keep up with the demand."
The pie for whom? For drug dealers who give power users their LLM fix so they feel smart and can fake it?
The pie is certainly shrinking for software engineers, as evidenced by the layoffs. Cocky startup founders may be next.
I’m a val.town user and townie has been really nice in conjunction with having stuff working and hosted right away, it hits the sweet spot for speed and flexibility. Tough call to make on whether to continue pursuing it, excited to see what you do!
Thanks!
Love the openness and candor of this post!
Thanks!