▲ 259 points
back
161 comments
This is really an argument about test-time scaling, even though the post never uses the term.
These days "test-time scaling" mostly means letting the model talk to itself for longer, but the first genuinely surprising results came from plain sampling. Google's AlphaCode generated millions of candidate programs and filtered them down to a handful of submissions, which beat the average human programmer in 2022, before ChatGPT even showed up.
Sampling is what AI is good at. Making examples and doing LeetCode are similar in that verification is clear and cheap. Compared to that, "proof" is still a vague concept, except where Lean works. See the fuss over the ABC conjecture. So humans are still needed.
The interesting question to me is what happens after enough learning from "sampling." Isn't AlphaGo's move 37 an AI's nose? If that happens in mathematics, we may end up with results that are correct, machine checkable, and not explainable in any way we find satisfying.
> Google's AlphaCode generated millions of candidate programs
The trick is avoiding the infinite monkey problem. If your problem is amenable to RL, then you probably don't even need an LLM, Monte Carlo Tree Search gets you there with less expensive hardware.
> Sampling is what AI is good at.
You might think so, but I tried asking ChatGPT to solve one of the puzzles from https://en.wikipedia.org/wiki/Countdown_(game_show) (which a Python script can brute-force on my 12-year-old hardware in half a second) and it made an elementary arithmetic error that's decidedly not human-like.
for somebody who's out of the loop: what's the fuss over the ABC conjecture?
> we may end up with results that are correct, machine checkable, and not explainable in any way we find satisfying
sounds like quantum mechanics
> A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that are difficult to stumble on by accident. It is hard to say precisely what would count as such a proof, but I think we’ll recognise it when we see it
Agreed. I find that after seeing these results from OpenAI we undeniably have a machine that has:
* General knowledge of nearly every subject humanity has ever learned
* The ability to simulate reasoning (albeit sometimes not very well) with that knowledge
* The ability to reference across the domains of knowledge
To me, this is more or less what I would think "Artificial General Intelligence" is. It's the cumulative knowledge of all general human intelligence, baked into an artificial form, which can then use that knowledge to achieve novel goals.
In many cases of mathematical breakthroughs there is an insight that comes from just happening to know a combination of already existing ideas and then combining them to solve that problem. This is where having that general knowledge seems particularly strong because we can run these machines for weeks on end effectively trying to brute force.
That being said, I could never imagine an LLM in its current form inventing something as elegant as the Fourier transform.
>> To me, this is more or less what I would think "Artificial General Intelligence" is
So then you need to explain ARC-AGI-3: https://arxiv.org/abs/2603.24621
"Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%."
Back 1996, EQP automatically solved the Robbins conjecture. But nobody concluded EQP was generally intelligent.
Would you trust a bridge built by it with zero human interference? A train? A plane? A skyscraper? What about fight in a war? Unless you can, I wouldn't consider it AGI, because you're actually trusting human intelligence to verify the bridge or train or plane or skyscraper is safe or that the robot is following orders. And even then, you're trusting human-influenced guardrails etc. I would consider it AGI when an AI-created LLM can do all of these things and you trust them with your children's lives. Would you trust an AGI cop to protect your children from a violent criminal? Unless you can, believing what we have as AGI is just an empty opinion with no meaning behind it.
IME, LLMs are primarily good at grinding through cases, which is why you see them pushing upper and lower bounds and finding counter examples.
I spent a few weeks working on a number theory proof with Claude off and on and it spent hours and hours and hours grinding through one shape of polynomial after another, reporting "progress", and it's true, it proved what I was trying to prove for more and more classes of polynomials, but it was biting off pieces of an infinite tower of classes with no hope of closing it for _all_ polynomials.
That happens to be a good way to find counter-examples, though, and when I posed a slightly different version of my problem, it found a counter example in about 90 minutes.
And in fact, finding the counter example for the related problem allowed Claude to finally prove the thing I wanted to prove to begin with, by lifting the problem to a characteristic where that counter example didn't exist, proving my question there, and then proving that it still was equivalent to my original question.
> I could never imagine an LLM in its current form inventing something as elegant as the Fourier transform.
This is the crux imo. It doesn't really matter what absolute capability AI has at the moment, but whether we are on track with respect to architecture and training approaches. In my mind the only test for AGI is, if the thing trained up to the cut-off of Fourier's time (or Descartes, Newton etc), it should arrive at or exceed their insight.
^ As a measure of intelligence. But it's probably the case either way that LLM is more valuable in terms of the coordinated grunt work we'll put it to, than leaps of insight.
I share the same thinking. What do you think is a good way to try to define this "elegance"? If we try to use the mental framework of
Step 1. LLM "brute forces" a search
Step 2. We train on this trace
Step 3. In the next model, LLM internally makes a "shortcut" for this path and "brute forces" it quicker (or one shots its in the best case)
And we want to ultimately show why that definitio evades this framework.
I would be extremely surprised if something as elegant, terse, and useful as the Fourier Transform had been missed by human mathematicians up until now. All expressible theorems are enumerable, after all (if we limit ourselves to a finite alphabet). It seems likely that any new theorems are long, highly complex and esoteric, regardless of human or machine origin.
That’s not the criteria outlined in the quote you just used.
For a list of AI accomplishments in mathematics see https://mathoverflow.net/questions/502120/examples-for-the-u... - or a candidate list here: https://aimath.robertj1.com/ . Many have observed an affinity of AI to the search for counterexamples - or examples. Looking at afore lists, something much more sociological crosses my mind: There is a hunt for answering prominent, clearly stated problems. I'm not a mathematician, but is this mostly what progress in mathematics is about? How about stating worthwhile problems in the first place? What about theory building? Am I right saying this is equally important, but none of those utilizing AI for mathematics seem to be interested in such?
You’re correct that those things are also what mathematics is about — but they’re less constrained, hence current LLMs aren’t as good at them.
However, your last question is incorrect: people are working on that, but there haven’t been hugely useful results.
But as an example, I’ve been slowly working on implementing frameworks for theory distillation — eg, take a corpus of science papers and derive a consistent model of the world from them, such as in Lean. (Or more specifically, a sheaf defining what consistent theories are possible.)
I know nothing about mathematics, but are there not famous mathematicians like Terence Tao who utilize AI and are obviously interested in theory building?
Given coding agent's demonstrated difficulties with concurrent code, even relatively simple concurrent code, it would be interesting to see how they do with temporal logic. I don't know enough to throw AI at the problems in that space but I wonder if they wouldn't crash and burn on it.
(I haven't had the opportunity to throw a current-gen frontier model at a concurrent problem because I haven't had one to try out lately. The best concurrency is no concurrency and the second-best concurrency is the "web request" model where many web requests are nominally running concurrently but they are otherwise fully isolated from each other and not trying to communicate at all. So maybe they're better, but I feel like if they were a lot better somebody would have noted that in a place I'd have seen by now.)
The harder test would be whether they can come up with the right invariants and abstraction in the first place
A thoughtful and measured post, as usual from Gowers. The final note is neat and worth pasting out here in full:
> A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that are difficult to stumble on by accident. It is hard to say precisely what would count as such a proof, but I think we’ll recognise it when we see it.
> with hindsight come to seem beautiful and natural.
…because they’ve been internalized. I wonder if the author has ever questioned where his notions of “beauty” and “natural” come from.
>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural.
I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating the capabilities of current SOTA models to implement an Agentic application for job search, here is some simple inhouse built evals I run today, since I don´t trust LLM vendors published benchmarks...
Models tested: GPT-5.6 Sol in Extra High mode and Opus 4.8 Max.
TASK REQUEST: Clear, not too long not too short prompt, for LLMs to go out and research freelance consulting gigs for one specific IT domain, and in one specific country in Europe, including maybe opportunities driven from temp agencies based in geographically close countries.
RESULT: Models go out, fetch the data, and completely misunderstand the task...offering on first results, permanent roles instead of freelance, and based on the country where the agencies are, not in the one it was request for. Think for example IT jobs in Ireland, while freelance agency in London.
ANALYSIS: No intelligence I can call it shown by models, adding cognitive effort for human in the loop to detect subtle factors, and therefore totally useless for agentic app...Best practices would be I guess to add agents on top of agents but although in the p95 of cases that will reduce the errors...for the remaining 5% that could have hallucinations or logic hallucinations like these ones, compounding on top of other logic hallucinations.
I dont care about the theorems being proven. At the end we will found out what most mathematicians were doing, was just exploring the same combinatorial and abstraction patterns. And because of that I am sure LLMs will make mince meat of a lot of mathematical domains.
But right now, what we call intelligence is not existing where it matters, and Ed Zitron is right its a parlour trick.
Since no one has mentioned it yet - just want to point out that Timothy Gowers is a Fields medalist.
That means the post was good, even though the blurry math images are illegible. But yes, as a Fields medalist he is an expert on LLMs.
Increasingly, I've begun to think of LLMs as sources of really interesting random objects: large pieces of "reasonable thinking" conditioned on a task. It's not that these are correct, in general, but instead they're a concentrated form of random search where that "randomness" is very likely to follow plausible, human patterns.
You can toss it at a task with a suitable machine for transforming that raw material into action and it'll rattle through and sample "plausible human behavior" at that endpoint.
There are more clever ways to use it, but a general tool here is to upgrade any sort of stochastic search to use this new form of random sampling. It'll be way more efficient, properly conditioned, because it just won't visit implausible things nearly as often as competing random sources.
If you want to peek inside how a model solves a math problem have a look at some data visualizations I made solving basic multiplication.[0]
I wanted to demonstrate capacity (how well it does a thing) instead of capability (which things it does, like drawing a pelican on a bicycle with SVG or solving a Rubik's Cube). To understand how LLMs solve math, look at the simplest case of multiplication. I deconstructed and classified the thinking token output. It is very important that model training yields thinking token output that structurally follows an observe, orient, decide, act (do the multiplication), and observe again loop.
You probably know it, but Boyd did not at all suggest speed beats quality. If anything, his realisation was the opposite: the US aircraft had a more open canopy and thus improved quality of observation, and that was what won despite the superior power and turning performance of the Soviet aircraft.
> If they were, then their big speed advantage over us would mean that there would be much more of a flood of results.
Is this true right now? Just recently Jarred Sumner tweeted [1] that he managed to make some progress on the Riemann hypothesis while on a jog. Managed to get somewhere by encouraging the llm to “keep going” and “believe in yourself”.
This raised a few questions for me. Had no one at Anthropic thought to try this earlier? It's an interesting footnote that a software engineer there pursued this. How many people in the world can actually verify a proof? How many would we need to sit around and do the right incantations to get a proof out of it? How many would we need to verify and give those proofs value and meaning? What happens when there are more proofs than verifiers? How many will be around in 100 years?
I think it just turns out that a lot this stuff is more socially useful than anything else. The 10 proofs drop came and went in the daily news cycle. Perhaps math is already in it's chess like "for fun" period. I am interested in when we find a very high real-world utility breakthrough math/physics, some space where we've already poured our best human resources at it.
[1] https://x.com/jarredsumner/status/2086869681785500011?s=20
The result in [1] is not progress on the Riemann Hypothesis. It's another fine result, but it's not progress on the Riemann Hypothesis. You could prove 100% of zeros are on the line and it wouldn't show the Riemann Hypothesis. 100% of natural numbers are composite in this same asymptotic density sense, and yet that doesn't prove that all natural numbers are composite.
The Vinogradov example seems curious. The "existential N" there is for encoding the intuition that the property holds eventually, e.g., that we don't know or don't care about which particular N along the number line for which P(n) is eventually true (forall n such that N < n). So in this interpretation, it is like using the quantified E like a Sorities/heap/vagueness argument, it is functioning metamathematically. Mentally I am picturing a number line, and then there's a vague area on the line where for everything to the right of the line it is colored "P". So why isn't this forall-exists usage a bit pathological, isn't it more that Vinogradov's theorem is "halfway" between either type of quantification? And that seems a very special case of how counterexamples break down rather than the general case? Could someone more mathematically knowledgeable explain this?
Correct me if I'm wrong, this is not the right way to ask this question.
LLMs are good at pattern recognition, so its less a type of math that they'll be good at, and more that when you provide documentation or text that can be easily parsed/compared to its training data/reasoning ability, the better answers you get from an LLM.
Also, you need to be knowledgeable at the same thing you are asking the LLM to do, to verify the answer it gives you (at least for the time being).
How i reason about this is that, there are two types of science works, exploitative and generative. Exploitative science is kinda like what ingen do in jurassic park, finding an use case based on available tensions in the literature.
Funnily enough, 'generative' ai is not good at generative science, of which requires unique human perception that is not purely symbollic manipulation, but requires a form of revelation. That I think is not here yet with ai...
It seems intuitive that finding a counter-example might be easier than proving a generality, since you're starting from a concrete goal ("build a foo that has properties X, Y & Z") that you can branch out from, identify sub-problems, etc.
Proving a generality seems much more difficult since you don't know what you are trying to build, although I suppose in some cases you can prove it by proving that it's impossible to construct a counter-example.
Disclaimer: I only scanned the article quickly; I might be re-stating something already in the article.
We have just got some very strong evidence about the way in which LLM-based systems solve mathematical problems and this evidence supports what many have already suspected including myself.
Here's what I'm talking about. On 10 August Anthropic released an article [1] claiming that:
An unreleased research version of Claude has improved on a longstanding lower bound for the fraction of zeros of the Riemann zeta function that satisfy the Riemann hypothesis. Drawing on extensive prior research by mathematicians over the past decades, it has increased this bound from 41.6% to 67.2%.
The same article describes the methodology followed by Anthropic's employee, Jarred Sumner, who prompted Claude, as follows:
Jarred Sumner, an Anthropic staff member (and non-mathematician), prompted Claude to “take a real stab” at the hypothesis itself, leaving the mathematical choices from there up to the model. Initially, Claude generated and tried 650 ideas, none of which worked. Jarred prompted Claude to try again, and it spent a day and a half coordinating about 60 Claude subagents, which this time went much deeper: between them, they ran 2,400 shell commands and wrote hundreds of Python scripts.1 The subagents ran thousands of numerical checks against known zeta zeros and refereed one another’s work. Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”).2 This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress.
Jarred got Claude to throw stuff at the wall repeatedly (650 initial "ideas" plus unspecified more by "60 Claude subagents" ... running "2400 shell commands" and "hundreds of Python scripts") and then kept whatever happened to stick. In this case, by happy accident, what stuck was an improved bound of the zeroes of the zeta function etc.
This is how every single mathematical result reported by an AI company has ever been generated. They throw stuff at the wall and take whatever happens to stick.
This approach works. Not only it works, it is, in principle, a universal problem solver. "Millions of monkeys on typewriters" will eventually produce a proof of the Riemann hypothesis; or a disproof of it.
The key point being "eventually". Is this a way to do mathematics research? Can that replace mathematicians?
In AI, this method is well-known as the "generate-and-test" method. It is ancient, basal to AI if I may be so bold. It first appeared to my knowledge in the Logic Theorist, the proof-finding program that Simon and Newell presented in the 1956 Dartmouth convention that named "Artificial Intelligence", to such luminaries of AI and CS as John McCarthy (the real "godfather of AI" who named the field), Marvin Minsky, Claude Shannon and others.
We've had the ability to brute-force all of mathematics "eventually", given "enough" compute for nearing a century now. Why haven't we solved all of mathematics? Are LLMs really so special that they can out-brute force search every previous brute force searcher?
Well, you tell me, HN. I say: no.
___________
It is often overlooked how expensive these models can be to run, and the false positives or other dead ends. You are going to be burning through a lot of $ if you use the latest models on hard problems, with no assurance of progress...
From my experience, you have to be good at math to trust an LLM to do the math.
There’s a well-known essay from the 90s where Timothy Gowers predicted that the creative/intuitive side of mathematics would be entirely taken over by machines before 2100.
The biggest win for AI dev efficiency is cutting down what gets loaded into context. Semantically matching tasks to the top tools helps a lot.
shameless self-plug:
two hard things in computer science
https://blog.est.im/2026/stderr-04
I think LLMs are reall good at conjecturing based on existing knowledge, but inventing new tools/lemma and new paradigm? Not really.
They are exceptional at the spending cost math :)