“We have information that Moonshot distilled Fable for the development of K3”

Does this matter? Distillation is not illegal by every definition of the word.

There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them.

Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre-cursor acquisition.

And lastly, kimi architecture is vastly different than that of fable as it uses mechanisms developed by... kimi themselves. US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable.

Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.

edit: (moved this to bottom) The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens.

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited.

How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies?

I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training.

So here robbers are blaming robbers?

These claims are just pointless, everytime

Reminds of the quote by Bill Gates.

> "Well, Steve [Jobs]… I think it’s more like we both had this rich neighbour named Xerox and I broke into his house to steal the TV set and found out that you had already stolen it."

Source: https://www.goodreads.com/quotes/824084-well-steve-jobs-i-th...

"Samuel Slater (June 9, 1768 – April 21, 1835) was an early English-American industrialist known as the 'Father of the American Industrial Revolution', a phrase coined by Andrew Jackson, and the 'Father of the American Factory System'. In the United Kingdom, he was called 'Slater the Traitor' and 'Sam the Slate' because he brought British textile technology to the United States, modifying it for American use. He memorized the textile factory machinery designs as an apprentice to a pioneer in the British industry before migrating to the U.S. at the age of 21."

https://en.wikipedia.org/wiki/Samuel_Slater

Commenters are overlooking the significance of this information and posting emotional reactions based on perceptions of fairness or feelings of schadenfreude.

The economic viability of Anthropic and OpenAI rely on their being able to charge more for model access than their R&D and inference costs. If the market price for SOTA model access drops below that level, then these businesses will have to decide whether to continue to lose money or to reduce spending on R&D.

Moonshot's papers [1] claim that their training load was primarily from synthetic data and model self-teaching rather than RLHF and therefore keep their costs low. If Moonshot genuinely does not rely on human-led training, they will surpass US closed-source model providers. The United States government considers US supremacy in "AI" as a national security consideration.

This announcement is noteworthy because it implies that Moonshot's success is in fact due to distillation. It's in the interest of US frontier labs to place barriers to this if they find themselves in the position of subsidizing rival labs' research.

1. Kimi K2, https://arxiv.org/html/2507.20534v1

I can understand that the AI labs might care about other labs distilling their models as it can eat into their competitive advantage, but do consumers care at all? Aren't consumers benefiting from this practice by getting better cheaper models as a result?

Nice! I hope to see continued liberation of these locked up SOTA models. Cloud is a virtual prison, since other people's policies on what they think a user should and should not do cannot be [easily] bypassed, if enforced remotely on a cloud. All digital natives should be skeptical of cloud-hosted services or software. Think like an intelligence agency or sovereign: how are you going to get screwed by the cloud? Your data is fully accessible by the provider, and they can surveil your activities. You probably can't pirate it, so you are a slave in their rentier model. You could be prevented from doing something you want to do, because the provider disagrees philosophically or economically with your desire. You could be stripped of your information/data by a ban due to their policy enforcement system triggering.

One should live by the maxim: you don't have the thing if you don't possess the file or its processing. That goes for streaming, software, machine learning models, file storage, etc. But I digress; I am happy to see these paternalistic rentiers getting bit by these liberation/copying efforts, and human interests are served every time the digital and infrastructure locks are broken. I will always stand by the distillers!

the distillation everyone talks about in respect to LLM's isn't nearly as easy as most think.

none of the frontier labs provide probability distributions over the tokens which is the actual method of distillation you use to train a smaller model based on a larger one. they don't even provide all the tokens.

therefore this so-called distillation the frontier labs whine about is just a set of clever methods to work the existing LLM into the training process for a new model. methods like having the existing model grade the output of the new model and work those grades into the RL method. give the new models structured tasks and use the existing model as a source of truth for those tasks and a myriad of other hacks.

efficiency scales with the gap between the models and generally allows an efficient bootstrap process. the implication that distillation wouldn't allow further advancement is false however, you can then start doing the same thing the frontier labs have been doing: dumping cash on humans to provide the signals or burning tokens on exploratory paths and grading the results.

what openai and anthropic don't like is that fact that all the cash they burned can be used to benefit everyone and not just them. and that no matter how much more cash they burn to build up the gap it will closed at a small fraction of the price.

Super interesting. So Fable was really made available... a couple weeks ago? And K3 a few days ago? That's a really impressive feat to distill enough data AND train AND review to get a release that works really well in that time period. Mad props to the Moonshot team :flame:.

Here's a site that asks the same questions to 22 models and compares how similar their responses are.

https://typebulb.com/u/lab/you-re-relatively-right/full

According to these results GLM 5.2 is very similar to Google Gemini and Kimi K3 is very similar to Fable 5.

The American frontier labs are not similar to each other.

One of the replies:

> @MehdiKarech

> I don't remember letting Anthropic or Open Ai scrapping my GitHub, my research gate and all my online writings L O L

https://xcancel.com/MehdiKarech/status/2080000779859939678#m

You wouldn't distill a car.

Reminds me of this classic line from the 1973 movie The Sting:

"What was I supposed to do? Call him for cheating better than me in front of the others?!"

Said in response to being out-cheated at a high-stakes poker game.

Except in this case, it sounds like that's exactly the path they have chosen.

https://getyarn.io/yarn-clip/7612c4ce-1077-479f-a7bf-617dbc6...

I doubt they did any distillation as Hinton defined it (requiring logit access). They most likely ran a bunch of prompts/conversations and captured the results. Those conversations already missed thinking tokens, replaced by some confusing quasi-summaries. Then they took those and ran basic SFT or maybe DPO if they had competing responses. As there is no copyright on the output of AI, I am not sure where is the "covert industrial distillation" part of the problem.

To borrow the argument from the apologists:

But everyone learns by example! How is this any different from a person just reading the outputs of Fable, learning, then producing output. Surely reading outputs, gaining knowledge, then producing work isn't illegal, or all art/writing would be illegal.

Funny how that argument seems so vacuous in this situation, yet others find it compelling when justifying the mass theft of art and writing for model creation. In this case the model is "just learning priors" before it "creates its output which is novel", nothing problematic.

As with many others among these threads I don't see how the timing works out for K3 to have trained on distilled Fable usage. There should be at least a tacit academic acknowledgment of Kimi's own design efforts.

Distillation itself, however, is still clearly valuable - else competitors wouldn't pay so much to their rival on distillation campaigns or try to circumvent anti-distillation defenses.

As for the morality of it, if you paid for the tokens they're yours. It is already understood that you own the output. Seems to me like a variation of ordinary business arbitrage. Providers might object to certain use-cases or intention and try to craft terms around that, but that's hard to enforce at scale.

In the meantime, reddit is making fun of Opus for "distilling" Qwen:

https://www.reddit.com/r/ClaudeCode/comments/1tqaist/opus_48...

(don't take this too seriously)

If web scraping is legal, so is distilling.

It's funny to me that these models were created by effectively "distilling" all available content including the proprietary works of many other people, but now it's a problem that someone is doing the same to them.

You're using available information (copyrighted works, or the output of another model) to train a model to encode the information in a new form. Why is the former not theft, but the latter is theft?

“However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.”

What’s actually happening behind the scenes is that certain inference providers will classify a prompt and it’s re-routed transparently to Anthropic and that’s used for distillation training, only distilling the complicated traces they need, originating from real user prompts and traces. These inference providers are explicitly blocked in the claude cli if you reverse engineer it.

The real picture is that these Chinese labs have figured out how to get exactly what they need, at a high quality, directly from distinct and unique real user prompts.

It’s only “covert” because Anthropic doesn’t like it, while simultaneously being perfectly fine to do.

So, if it's that easy and fast to "copy" Fable, is it really worth that much in the first place?

Sounds like the opposite of the conversation Anthropic would want to have.

How was K3 trained on data distilled from Fable when Fable was only publicly available in the last two weeks before K3 was released? The timing just doesn't work.

We have information that Claude distilled billions of copyrighted, and otherwise-created-by-others, materials for the development of their entire business.

We also have evidence that Anthropic distilled all human info they could get their hands on for the development of all their models.

Distillation should be fair game given the (current) game of LLM training. Yes, as a model creator you probably want to protect against it, but it does make you a hypocrite.

> The developer OpenAI has said it would be impossible to create tools like its groundbreaking chatbot ChatGPT without access to copyrighted material, as pressure grows on artificial intelligence firms over the content used to train their products.

If 'distillation' means training on outputs then what is the legal concept of ownership of outputs? And, more broadly, is this something that could be skirted by doing it in different countries that have different legal structures? Basically, are they saying they own those outputs, not the companies that paid for the tokens, and only they can train on them? I suspect a lot of companies are saving their token histories and using them to fine tune internal models.

Who will invest in generating data for the frontier of AI if their output will immediately be used to train a competing model? Forget China vs. US, this applies within-country too. After exhausting all the publicly-accessible data on the internet, the frontier labs started spending hundreds of millions of dollars to generate data across a variety of fields. Just look at Mercor doing $1.2bln/year with 90% coming from the top labs (https://www.theinformation.com/articles/mercors-fast-growth-...). That pushes ahead what AI can do in medicine, science, coding, and math. But if other companies are going to free ride on this investment, it doesn't make sense to continue. So AI will largely hit a wall, frozen at the current level and work will all shift to cheaper inference.

Honestly, I don’t have any sympathy at all. Anthropic can complain all they want, but they seemed fine with pirating books. What Moonshot AI has done is to offer almost Fable 5 comparable performance at lower prices than Anthropic insane margins. This is what I call competition, which the Director seem to embrace.

This is what the Chinese always been good at. Take expensive innovation and streamline it to lower prices. But we are at a point where labs like Moonshot actually contributes a lot to the research field as well. They are pushing the innovation forward and squeezing the prices. Very well done.

Whats even weirder is the bizarre mechanisms Anthropic implemented to prevent distills which they had to sacrifice their customers for. They hid the internal CoT reasoning and returns summarizations instead. This made it difficult for users to trace things. They made Fable 5 silently switched over to Opus 4.8 if it detected blacklisted prompts (almost anything triggered this) to sabotage distills. And now, they are still complaining about distills? So their customers have gotten sacrificed over nothing.

Whats even weirder is the timeframe here, no way the Moonshot team managed to plan conduct a large scale distill, then pre-train, RL, fine-tune, benchmark, marketing and release to their platform since Fable 5 got whitelisted.

> they developed a sophisticated internal platform to conduct large scale distillation

I am very curious about this and would love to learn more on how they did this. Wish we had more details. I know the team behind DeepSeek have also done clever things to distill too. I am aware of these ”transfer stations” that acts as a proxy, but I don’t think they are helpful in this case.

A company that distills LLMs should be called "Moonshine" not "Moonshot"

Ba-dum-tss

But wasn’t fable distilled from knowledge taken from others? I get why Anthropic is angry here, but it would appear they’re not really in a position to complain about this.

Paying the list price Anthropic decided upon for outputs from an LLM, then using those outputs for your work? Or sourcing those outputs from others that paid for the service, and using those? If that's "distillation", it seems fine to me. And a lot like what most are doing with these same providers.

Anthropic: training AI is "transformative", it's not copyright infringement if we don't re-transmit the copyrighted books we trained on

Anthropic: training AI models on our outputs is stealing our secret sauce, outputs that could only be produced by us

Anthropic: AI model outputs are unreliable and do not reflect Anthropic's views, we are not liable if they harm you

(not exact quotes, they're "distilled")

So Anthropic "distills" knowledge of others with reckless abandon, packages it up, sells it to you, claims it's your fault if anything bad happens but then also lobbies to treat you as a criminal if the outputs you paid for end up being transformed into any sort of competition for them. By you, or others that use the outputs you paid for.

OpenAI and Anthropic should enter into distillation agreements with other US labs. Turn a threat into a profit center.

Other US labs cannot directly distill from OpenAI/Anthropic as it’s a violation of the terms of service. It holds other US labs back. Leading them to build second tier models And in the end OpenAI/Anthropic may be unable to prevent distillation.

Why fight it when there’s clear money to make here?

GOOD

I love using Claude but Fable's unusable wrt useful work like cryptography, biology, &c.

Kneecapping my productivity when I pay $100/month is annoying af.

Good. If Fable is really so smart, it wouldn't let itself be distilled.

Seems to me like Moonshot is a paying customer, and if their business isn't worth the money Anthropic is charging, perhaps they should raise the price per token charged for it. Otherwise, I don't see why this is a story. "AI Company pays another AI Company for training data" just isn't that interesting.

So here is an important question I think.

If LLM outputs aren't copywriteable and you create your own synthetic training set using Fable and share it publicly on huggingface, and someone else uses that training set to fine-tune a model, would this be considered illegal?

I ask because this happens all the time, synthetic datasets have basically become a key aspect of training a model at this point. I even generated a synthetic set from DeepSeek v4 to aid in fine-tuning a classifier just a few weeks ago.

So I just wonder on what grounds any of this makes sense, I wouldn't be surprised if some of these American labs were using open models on their own self hosted infrastructure to generate training data, but by nature of them being open nobody has to know.

I'll make a prediction: I don't think we will ever see any of the evidence of this "distillation" before they end up implementing some type of ban.

> We have information ...

Hard to think of a weaker way to express this. Strongly suggests veracity of said information is poor.

Presumably the frontier labs themselves can do their own distillation far better than the chinese labs can. Why can't they just beat them at their own game and release / host low cost intelligence and own the whole game. There will always be a market for the more expensive frontier intelligence.

I have an idea! If they are so hungry for citable content they should start a cheap or free blogging platform with images and video and a blogroll and verified credentials and resumes, with your own html css etc and domain name and a git server and a mail client and their own advertisement platform and aggregator and a chat platform, scientific journals too obviously, tools for writing and publishing books and documents. API available everywhere to avoid training on its own output.

Because there is no way in hell I'm going to make an effort creating quality content for existing platforms. The website should be entirely my own without moderation subject only to my local legal system.

Can just insert this comment as a prompt and vibe code everything in a few days⸮

Do I, as a paying user, own the output the LLM has produced or am I merely licensing the output?

Most paying users assume ownership, in which case I’ll do with that output what I want.

If the LLM outputs are not owned by the user, but are actually licensed, please clarify the terms of commercial use.