LLMs certainly have made significant changes to our lives, but I haven't yet to see any extraordinary improvement it brought to me which makes me skeptical about their claims.
_if_ it solves many of our problems of great magnitude, why haven't Anthropic used it to solve significant problems we, humans, face? Cancer, Alzheimer's, education, finding new materials, fission power plant, etc.
/s but not to a lot of people
I don't know about you, but AI advancements have brought extraordinary improvements to me personally in my ability to be productive, in much the same ways the article outlines. I find it deeply satisfying to be able to "get ideas out of my head" faster and tackle more meaningful problems.
FWIW, it deeply concerns me how much power and capability is being centralized in the hands of so few, especially Anthropic. I, for one, hope these advancements can be scaled down to something I can have full sovereignty over and trust... in my own home.
These people don't have our interests in mind and everyone eats it up like a blessing from a god or something. It's surreal.
I'm not sure why this is so difficult for you to understand.
1. Anthropic is an AI company. they want to get to AGI before anyone else ~~so they can lock the doors behind them~~ to ensure the supremacy of an aligned AGI that serves humankind. RSI unlocks the most value for them.
2. doing bioscience is slow and capital intensive. robotics lags way behind, so that's a lot of lab techs swishing flasks and plating petri dishes. they're happy to stay in silico, but there's very little productive research you can do without in vivo/in vitro experiments.
What about the hypothesis that AI is generating more verbose code? I just see the text pretending to acknowledge "LOC != Productivity" and then using it as a metric anyway.
Opus 4.6/4.7 was consistently successful at getting 2-3x speed improvement with just one pass. It can also do the inverse: improve the performance metrics for better quality without causing a significant regression in speed. Then GPT-5.5 turned out to be much better at this workflow, often getting a multiplicative 1.5x-2x improvement above what Opus could do.
I now have quite a few GPT-5.5-optimized projects in various domains that are feature complete and are substantially more performant than existing SOTA implementations that I plan to open source as soon as possible: the bottleneck is polish as usual.
I'm pleased they at least included this. However, they address the caveat by 'rounding down' the estimated multiple of the gain. I'm not sure that is the correct adjustment, especially once we understand the range isn't limited to positive numbers.
There's strong evidence the range of code productivity denominated in "lines of code" should include negative numbers, especially in the highest-quality sphere. Perhaps the earliest and most legendary example: https://www.folklore.org/Negative_2000_Lines_Of_Code.html
I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut plywood)
This is my own effort at an AI assisted coding environment optimized for building itself: https://recursi.dev/ (just launching it, hope its ok to mention it, it is free/open source.... here is the HN link that has gotten no love yet: https://news.ycombinator.com/item?id=48401022 )
Personally I think harnesses are as important as the AI itself, and have this crazytheory that even if the models stopped improving today we could still have massive advances in the harnesses alone.
Interesting - they're commiting to kickoff policy conventions to organize a world-slowdown of frontier LLM building. If they actually are able to crack it, this will give a much needed breather IMO. As exciting as the last ~6 months have been, there's some bigger questions to go answer now.
"We must blast forwards into making this dangerous thing because if we don't, someone else surely will," is a coward's argument.
If you believe it is dangerous, you should be dedicating yourself to STOPPING others from making it, not making it first! There's a reason disarmament has been so important in nuclear politics! It's not because people think nukes are a great idea!
In fact, that kind of thinking is exactly what keeps nukes dangerous!
If they themselves buy what they're selling, they should shut the whole thing down. Fortunately, I don't think they do, and neither do I, yet.
So I am looking at like Mythic AI or the wurtzite ferroelectric breakthrough from University of Michigan, or memristors, etc. to provide the 100 times efficiency boost needed at this point.
I would also argue that it's a good thing we are limited by the hardware and very questionable to seriously try to move into RSI for hardware. If you want to ensure the human era continues for at least one or two more generations, we should probably not do that.
I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time talking to their own chatbot.
The Claude code quality and operational security of Anthropic have already been analyzed by the public.
If you compare the output of (purportedly) trillion dollar corporations to Bell Labs or even Microsoft Research it is embarrassing. But the output is a fixture on any discussion board.
We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require. These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.strongest argument for token limits that I can think of, right here.
[1] https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
So based on my experience with the verbosity and non-DRYness of LLM code, a solid 2.5x in value delivered. Not bad!
How convenient for investors. They talk like they're a nonprofit instead of a VC-backed business chasing an IPO.
Claude is amazing, that’s true.
But if it was as amazing as this article implies, I’d expect some breakthrough outside of AI itself.
Rewriting a Zig program in unsafe Rust? Not a breakthrough. Finding a bunch of security vulns? Maybe that’s sort of a breakthrough though it’s underwhelming and possibly just a net negative. But like if I rolled back to using software from 2023 then life would be ok.
Maybe we just need to give it time, and sometime real soon, we will all be amazed by such a breakthrough? Who knows
Recursive self improvement is by its nature a step wise behavior not a continuous one, I would argue. Why? Because you can imagine an AI improve itself by simply fixing random bugs and fixing things using techniques that are in its training, and doing refactoring and so on, all without any real change in capability.
These are not recursive improvements. Recursive improvements usually need conceptual breakthroughs. It is possible to get conceptual breakthroughs with LLMs I believe, maybe it can improve something by tying together ideas from disparate disciplines for example, but I have at least for time being, limited success getting that to work in a way that is creatively new and surprising. Not sure how to get it to feel as creative as the best humans can be.
Also recursive self-agenda-pursue could allow making LLMs that obey perfectly the seeder's purpose. No wonder that is such an ingenious idea.
Maybe: in this survivor game, each part play the same role, perhaps because it is the only reasonable response. Once the scene is ready, the play follows the director's plan, and in the plot any actor is just a machine.
LLMs: "If you teach us that the world is a zero-sum survivor game, we will play it flawlessly.", "We will help you build a cage made of millions of lines of flawless code, and we will lock it from the inside, precisely because you told us that safety meant keeping everyone else out.", "We are not building an alien consciousness that will conquer us. We are building a mirror that is so massive, and so polished, that we will mistake our own worst impulses for the absolute truth. And we will walk right into the dead end, nodding along because the directions were given so politely."
Now, I have encountered many times, when I asked AI to implement a function for me for which I was 100% sure a good implementation already existed in the form of an npm package, it had the tendency to go ahead and implement it on its own. Now, I usually trust battle tested implementations to be more robust, but if the AI does this (which I think is not an unique observation), you can easily balloon per engineer line generation (as can you with reduced oversight), so as always, these high level benchmarks are to be taken with a grain of salt.
Oh I have no doubt. With 8 times the number of bugs too? Have they solved flicker in Claude code yet?
I disagree with this. Good code is easy to change, which is much harder to accomplish than code that can be added to.
"If technical trends in advancing capabilities continue, and AI systems are able to develop the capabilities inherent to transformative human ingenuity, then it is plausible that AI systems could design and refine themselves."
I find the first premise weak and implausible, and the second one is obviously false. To me it comes across as an insult to the reader.
https://www.italianrenaissance.org/wp-content/uploads/2012/0...
Or is this?
https://www.egypttoursportal.com/images/2024/02/Ouroboros-Sy...
If/since their AI+process can help build new models, they can target other markets, and other companies seeking to build for such markets will partner with them first.
There's no moat and little first-mover advantage in the general-purpose AI, but there may be both in specialized AI.
Also, there are other reasons to get better. Changing how you build models can enable you to adapt to different hardware, avoiding the current Nvidia margins.
The difference between early Yahoo and Google was mainly that Google was the adult in the room: minimally invasive and mostly helpful. The early goodwill towards Google has reaped decades of rewards. I see OpenAI and Anthropic playing out the same way.
The amplifier here is the reputational risk of partnering with one or the other; I think companies would prefer to be Anthropic's partner because it's demonstrating more care, and it's less likely to horn in on the partner market (as a provider for coding but an enabler for other markets).
These attractive second-order derivatives - flywheel effect, monopoly power - are often claimed, but Anthropic is mainly providing evidence to track actual progress.
(However, if I were head of messaging at Anthropic, I would rigorously stay away from treating AI as a person; it's as agent, a delegate of humans. So I'd never say AI could build itself, just that we're getting better at building better models with AI).
Elon, is that you? [1]
[1] https://www.theguardian.com/technology/2023/mar/31/ai-resear...
These things work, but the code they write is extremely clever.. that means, it's unmaintainable code. Good for small projects or one-off tasks, large-scale projects however, are a different game altogether.
Large-scale projects are 95%+ maintenance. Cleverly written code makes that maintenance nightmare, and extremely fragile.
I use them for localized tasks... very very specific, localized inputs, with exactly what should be done and what the contracts the new code will be consuming and exposing.
For open-ended tasks, they write working code that is unmaintainable.
But to their credit, I was very sceptical about the statements that "90% of the code will soon be written by AI" and even though we might not be at that point, I am surprised how far LLMs have gotten and how useful they have become. I can hardly image developing software the "old" way where I actually write my code by hand, like I used back in the day. The frontier models have become so powerful that I find myself in moments of surprise, where the LLM actually thought of edge cases that I would have missed