▲ 94 points
back
91 comments
If they succeed here, won't they have to gate access to this feature, too? For the same reasons as, "if I so much as mention mitochondria, it downgrades me to Opus."
Step 1. Make it so Claude can do anything — the whole point of AGI
Step 2. Wait, if the user can do Anything, that would be Very Bad!
Step 3. Err on the safe side with blanket bans of entire fields
The latter actually seems to me a sensible reaction to e.g. the compartmentalization used in the large scale cyber attack using Claude last year. Where they were able to do Bad Thing by dividing it into many, many Small, Seemingly Harmless Things.
Gated access sounds bad (and I agree it sounds bad!) but it might actually be the only sensible response to such a set of conditions. I'm not sure though.
--
I saw some studies recently which showed LLMs provide much more detailed information to expert users. So we can distinguish between competence and incompetence based on use of language, and that is a reasonable metric for harm reduction.
But I don't think we can reliably detect "user has harmful intentions", at least not at a sufficient level of sophistication of the attacker.
I just wait until a Opensource model inhaled all the chemistry books and papers. Lol.
They are following closely and the best offer 80-90% of the performance and come with a very small fraction of the costs.
Why wouldn't the US gov. outlaw the open source models?
There are no open source LLMs, only LLMs you can download.
No they don’t.
"Making Claude a chemist" is a punch in the face of all us here after we got a taste of Claude fable blocking almost any STEM-related topic. So i don't know what Story Anthropic wants to tell us with this blog. Probably more like "Hey, look at what awesome tricks Claude can do which of course you will never experience because you're not american, not friends with the CEO, not rich or simply because fuck you"
I think the story they want to tell is "we're not just taking your jobs, we can do positive stuff like Chemistry too, just like DeepMind!"
Except of course they mostly can't because Chemistry is about structure and molecular dynamics, not people's descriptions of experiments.
DeepMind if so far the only AI/AGI company - the rest are LLM companies.
It's a good marketing strategy though, since nobody can disprove that Claude is the best chemist :)
I think one time I asked opus about copyfail when it just came out and it did treat me like some sort of criminal, but are there really people that run into this on a regular basis other than cybersecurity experts (which cannot be a big enough group to generate all of this criticism)?
I have been trying to work on debugging tools using Opus 4.8. As it turns out working with low-level techniques that inspects and alters behavior of other processes is bordering close enough to cybersecurity research that it often hits the guardrails.
I asked it to clip a video onto segments based on location of spoken words (using separately done audio transcription) and it flagged that as cyber.
Fable wouldn't even explain what an amino acid is.
Maybe? It seems like their strategy is to accept that safety is imperfect, err on the side of over-triggering, and iterate.
I don’t think it’s a black and white “if fable 5 over triggers on bio safety in 2026, that’s the final pattern we should expect to see from post-Mythos 20 in 2036”
Providing access to the public is presumably just a temporary stopgap for these companies until they have robot armies to replace their customers and employees.
It would make more sense to have specialized versions of Claude instead of having a universal one ie BioClaude, ProgrammerClaude, NuclearClaude, CyberSecurityClaude, et al
This would be both safer and less annoying to use.
That seems bad because it kills interdisciplinary science.
The non-professional side of Organic Chemistry is one place where I think AI would really shine.
Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
But the synthesis plan is only 10% of the actual work.
The gap between writing down the synthesis step and actually doing it is also extremely large.
Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
The Ritonavir episode of Veritasium is a great example of how all chemistry on paper is a mere shadow of what actually happens in real life.
> Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
I remember two years ago, when I actually got into using graph data structures, wondering if maybe the "space" of available reactions for any given starter and target molecules could be mapped as a graph, with intermediates as nodes and reactions as weighted directed edges, so synthesis becomes pathfinding through chemical space.
Turns out, it’s a thing! [^0]
Edit: Makes you wonder how much interesting stuff is sitting in plain sight, waiting for someone with the right cross-domain awareness / knowledge / whatever to notice it.
There is a lot of graph theory in Chemistry - modelling chemicals as (vertex/edge coloured) graphs, reaction networks, etc.
Of course some molecules (eg aromatic systems, like ferrocene) are not naturally representable as graphs. I wonder if it is the same with synthesis - are there reactions hard to model as a graph (or petri net or whatever). One simple example I know is that you have to be careful with including a node for 'water' as it gets connected to everything else! Or at least in biochemistry it does.
Hamilton Morris and his stuff on clandestine chemistry is super interesting in this domain. Sometimes the chemistry is straight forward but access to certain chemicals is hard, so the procedure must change based on what's available not necessarily what's ideal
> Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
Sounds a lot like vibe coding lol
Organic chemistry seems like a discipline better done by chemists than forward deployed staff with their payoff function sharply truncated at an IPO which at this point may or may not happen on schedule.
Combine it with automated lab like this[1][2][3][4][5] (and many others) and it will iterate much quicker. Some already do but at a smaller scale, AFAIK.
[1] https://www.ginkgo.bio/autonomous-lab
What good is it if you can't use it? Or worse, if you can but it silently sabotages you?
> We measured three Claude models (Opus 4.7, Opus 4.6, Sonnet 4.6)
You can use those and they probably won't intentionally sabotage you.
Waiting for Claude to end up on Derek Lowe’s list of things he won’t work with
I feel like chemistry is one thing that current models will struggle with for the next while, because it's inherently 3D. In the micro world, shape = function. Maybe enough textual patterns will let it under chemistry, but like how do you describe a hydrogen shift without showing how it moves positions and rebalances bonds?
I think this is a very interesting concept/question. I feel like programming is more about shapes than anything else… but they seem to have mastered that fairly easily… but I totally get your point!
I work a lot with the geometry side of computer vision (camera calibration, 3d reconstruction etc.) and LLMs have been really bad in this space. They throw stuff at the problem until a minor improvement is made by over fitting on a testset, then gaslight that this is the best thing possible. Then I go do something like working with raster-based data, or some JavaScript based visualization, and it goes super smoothly.
Agree here, chem may be more complex than language, and not as definable as language. I think this is the realm that physical ai will overlap with.
I wonder if chemistry skills will be an emergent behaviour of world model research.
I have the opportunity to try this on some new NMR data on peptides. I’ll share everything on GitHub and show how it went :)
Curious to see how the others actually do.
I guess the future is one where every idiot has access to a genius servant, and all that implies
Fable pre-ban wouldn’t continue an old conversation about brain chemistry, sooo I guess they’re dusting off old tricks now that their new robot is offline.
Are there organizations or individuals using AI to solve world problems if they are so powerful as these companies are saying?
everything from amd in robotics space
Let’s ban this before it gets too powerful !
I don't think it should be outright banned, automated intelligence is like a gun, hammer, knife, nuclear warhead, etc.
I wonder if every future announcement will be met with this criticism. Like, what's the point if they'll ban it in a few days.
I don't really want the average LLM being able to tell anyone exactly how to syntheize lethal nerve agents.
Predicting 1D spectra from structure might be a problem better suited to quantum computers.
inb4 someone calls Bessent to explain how this can be used in fentanyl production.
I know your comment is about the absurdity of such a thing, and maybe the stupidity of the administration, but for the readers at home:
Fentanyl is actually a very simple molecule. The way it's (and other drugs) production is controlled is by controlling and monitoring the precursor chemicals. So if someone goes out and buys a lot of proprionyl chloride, they will pop up on the DEA's list.
This is a much more sane thing to do than hemming and hawing about how dangerous the AI chatbot is and placing yourself into a position superiority over your users, pretending you know better than them, assuming that you must protect them from certain knowledge for their own good, etc, etc.
Why does chemistry require inference?
I vote for "Claude Pinkman" as a name
"I'm sorry Dave, I'm afraid I can't do that."
“I can’t tell you that, you might hurt yourself.”