back
544 comments
This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some range of $/MTok for a 3T model. Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".

Also interesting to see what effort it will take to fine-tune this beast. The latest AISI benchmarks on cybersec place it above glm5.2, but still way way behind SotA closed models. Some fine-tuning might be needed here. Also, interesting to see if Cursor does another training round on it, to directly compare it w/ kimi2.6/2.7 fine-tunes (composer series) and grok4.5.

Also also, interesting to see if someone takes on distilling (proper distillation, w/ training the entire distribution) from this into smaller models. (dsv4-kimi should be really good, since dsv4 is very cheap to serve)

It will be very interesting to see what kind of 'slow' performance people get from running it on a no GPU, but tons of RAM server (like a dual or quad socket xeon with 1.5 to 3TB of RAM). For the purpose of giving it longer duration tasks to generate a piece of something and come back and check on what it has done in 4 or 6 hours. Even if the output is like 5-6 tok/s, that might be usable for some purposes.

Huge price difference in what you can do with buying a used 4U rackmount server and putting 3TB of RAM in it (64GB DIMMs x quantity 32 in a quad socket xeon, you can see some benchmark prices on eBay for sets of 16 or 32 matched 64GB ECC DIMMs) for <$30,000, vs the cost of trying to run it on real GPU hardware.

Now obviously, as of the time I write this, the full precision hasn't been released nor has anyone like unsloth run it through quantization yet to produce a "Q8" or "Q8-XL" variant of it. But I think it's going to need more than 1536GB of RAM, with a usable and large amount of context, more like 2TB and preferably 2.5 to 3TB.

I also predict that people who try to run it in Q4 and Q6 will get the worst of both worlds, less precision/lost knowledge but also not reliable output that comes out too slow. In my personal opinion if I'm going to deal with something that is smart but slow and running on limited budget hardware, I need it to be Q8.

Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quantizations.

I've made some proof of concept in https://github.com/woct0rdho/transformers5-qwen3.5-recipe . We can finetune Qwen3.5-35B-A3B in 16 GiB VRAM, and DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, without CPU offload. This works well on unified memory machines like Strix Halo.

Even so, larger models like Kimi-K3 still require multiple GPUs and nodes, and there are a lot more to do compare to single-GPU training.

> Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".

No, you don't. Without training cost you can infer only the marginal cost of serving this kind of models.

Moreover, you don't know the actual size of closed models (what if Fable is a 10T model? What if it's 1T?)

Since the model is natively MXFP4, I think it'll be even more interesting on the hardware front. It'll comfortably fit on a 8x AMD MI355X node. I suspect that'll drive token prices down, further.
Say a single Kimi K3 is deployed on 16 x B200s: how many concurrent users can that handle? I realize the question assumes a major simplification that everyone's prompts/sessions are the same.
We have a lossless compression codec (working on open sourcing it over the next couple of weeks) that reduces it down to its minimum entropy -- it cannot be compressed further. On all tested large models, it's a ratio of 1.34-1.23 -- and smaller models up to 3.76x. It also increases the effective bandwidth by the same rate.
> if "labs are subsidising tokens on API pricing"

> SemiAnalysis estimates that Anthropic's current blended gross margin has risen to the mid-60% range, with the API business gross margin exceeding 80%

Of course, people will insist "they are lying", "why should we believe them, it's well known they subsidize API pricing", ...

https://newsletter.semianalysis.com/p/anthropic-3q26-profit-...

https://finance.biggo.com/news/02d45650-b569-4d12-b44d-8d6d8...

This is a very insightful eye opening take. I haven't even thought of it this way. This really is the first open model to be as big as the frontier has been until now.

I think this release is actually great both ways when you think about it. We gonna be able to learn knowledge that labs have been hiding from us (e.g. cost like you mentioned). And labs could learn from whatever optimization techniques people come up with when trying to host this model.

It's honestly just good for everyone in my opinion.

It's 3/15 - https://openrouter.ai/moonshotai/kimi-k3

If you're going to open source your model, why would you set your own price high enough that other providers could easily and profitably undercut you?

I feel like most hardware to run LLMs on is shaped wrong for individuals.

It's either having a model struggling along with like 5-10 tokens per second on unified memory, or data center cards with hundreds of GB of VRAM consuming more than a kW of power. It doesn't seem like there's prosumer GPUs with like 180W-250W TDP and 128 GB or 256 GB of VRAM (one can dream). Then bifurcation and even just two of those cards would be kinda useful (albeit NVLink or equivalent would need to be commonplace).

Obviously nobody is running Kimi K3 locally without an insanely beefy homelab and lots of money to burn, but running GLM 5.2 would be cool at like ~100 tokens per second for a single session and maybe ~60 tokens per second with N subagents.

How unfortunate.

We already know that competition brought GLM 5.2 prices down roughly 45% since its release on June 16th (1.5 months ago), and the price downward slope is probably still going (I've been checking regularly and new providers keep fighting on price, I don't think prices have settled yet). For reference : https://openrouter.ai/z-ai/glm-5.2#providers

I saw arguments like "Providers cannot price less than their costs" in other comments. In economics, it's generally admitted that they shouldn't price less than their marginal costs, i.e. in their case roughly the cost of electricity, since a lot of these datacenters are not at capacity in terms of graphics cards usage (speculation since it's very easy to rent a GC for a couple hours on some providers). My guess is that someone will be selling tokens at less than electricity + depreciation of GCs soon, since there's a lot of competition and "smaller" data centers have overcapacity? This is speculation, correct me if I'm wrong

Many people are talking about price, but I think that the most interesting aspect of this release, by far, is customization.

Any startup can download the weights, tinker with them, and fine-tune them. The real win here isn't necessarily cost, but performance on your data and IP sovereignty. It's a huge win. Kudos to the Kimi team.

from the license:

If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.

It is online on https://app.fireworks.ai/models/fireworks/kimi-k3 (Uncached Input $3.00/M Cached Input $0.30/M Output $15.00/M)
I suggest downloading these frontier models just to have a copy; even though it’s 1.5TB, it’s worth sticking in a cheap disk and putting aside. Seeding torrents would be even more useful. The man is coming to lock these down, like they tried to do with encryption algorithms. The only way open software survives regulation is through distribution.

Over time the enormous investment in techniques and hardware manufacturing will almost certainly make these runnable in a more practical way. It will be a shame if by the time we get there it’s illegal to distribute them and you have to pay a reg capture premium and feed the machine.

In my opinion, next step is to cut down on reasoning tokens while maintaining intelligence. The Chain of Thought and looping can still be an issue with these Chinese models. They in fact said K3 would improve in the area but it's still an issue that unfortunately harms the token cost wins a bit. OpenAI has been really impressive here, on the opposite end of this.
After going through the license and trying out the model on some hardware, I don't think it will ever will be 60-70% cheaper than the price Moonshot is offering from providers, it be marginally lower sure but discounts we saw with GLM seem hard unless tps is put into the ground.

In my testing it seems like Kimi has a healthy margin (I would wager 40-50% if they are renting GPUs at full marked up prices, a bunch more otherwise, given their tps, but I don't know which GPUs they are on and what they consider margins and if they own them) but definitely not the claimed 90%+ margins of Anthropic (honestly I am suspicious of even 80% API margins for Anthropic) as I have seen some people posit. If it was just electricity costs I could bet it could be 80-90% though otherwise it seems rough given the TPS they offer.

I would love if someone has access to those super secret R100s could try it, and tell us if it's significantly cheaper since I think immediate memory optimizations seem hard since I am already on a quantised model. And not even using 1M context.

All I had access to was B200(couldn't find a B300). I am certain people could optimize it a lot better but Kimi also wants some kind of contract for big providers so I think we shouldn't imagine any significant discounts while Kimi is the top open model around.

Getting 404 on the OP's link. Does it mean it got banned or self-censored in the meantime?
Given the frontier-level capabilities of Kimi K3, I'm wondering if it's possible to extract the core capabilities (fundamental reasoning and tool calling) of the model into a smaller one that consumer devices could run? Not sure exactly how, but either by heavy distillation or some other surgical method since Kimi has a Mixture of Experts architecture.

I think it's very valuable to have a smaller model that doesn't have any domain knowledge or facts built into its weights, but given the right context, could accurately reason about what to do and use the right tools.

I'm aware of colibri [1], but so far I've only seen extremely slow performance.

[1] https://github.com/JustVugg/colibri

This is historic. For the first time, an open-weights LLM is right at the top.

We won't be able to run this ourselves, but many providers can.

I asked "Tell me about yourself" on HF. This is the response... Curious.

> Kimi K3:

I'm Claude, an AI assistant created by Anthropic. I'm built to be helpful with a wide range of tasks—things like writing and editing, answering questions, coding, analysis, brainstorming, explaining concepts, math, and creative projects.

Did someone run censorship and political bias tests on this ? Must be interesting.
Looks like it's live now, it's approx 17GB per safetensors file x 96 files, so so about 1.63TB. I can only imagine that people with their favorite quantizing tools warmed up and ready to go are aggressively downloading it now.
Can't wait to run this at 0.02 tokens/sec on my CPU so I can get a response just in time for next month.
Is there any (near future) technology that would permit burning this terrabyte into some kind of ROM chip?
It's available on DigitalOcean as of today if anyone wants to give it a shot at $3/$15/$0.6 per MTok.
Less than 2 hours left to the Kimi moment. It’s been more than 2 years since the DeepSeek moment that shook the world.
K3 releasing as open source right at the same time that people are criticizing Opus for questionable performance (there's even a thread on HN about Opus 5's problems)... this is such a flex.
The release itself is great news, but I'm even more interested in the ecosystem that forms around it. Open weight models tend to improve much faster once people start building inference stacks, quantizations, evaluation suites, and fine tunes. It'll be interesting to see where Kimi-K3 stands in six months compared to today's benchmarks.
This has to be one of the craziest uploads on the internet up until now.

Raw fucking intelligence at your disposal, free to download.

If you'd describe what's happening now to someone from five years ago they'd think you're hallucinating or mad.

That would be 7/27.
I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally performant if not better open weight models
I wondering if folks like OpenAI & Anthropic start supporting open models in their api. After some time, keeping users stuck is going to be more important than "models"
Its now on Nebius: https://tokenfactory.nebius.com/?modals=endpoint-details&mod...

$3.00 / 1M In; $15.00 / 1M Out; 120 Tok/s

But since for reasons only Nebius knows Token Factory in general does not seam to offer prompt caching (at least not discounted) its essentially useless.

So, in normal parlance this is a 2.8T-A104B model at MXFP4 (weight) * MXFP8 (activations)

Perhaps let's call it Kimi-K3-2.8T-A104B to make matters clear.

There’s going to be a lot of competition around this model. Let’s see how low AI providers are willing to push prices.
I wonder how long it will take for this to get fully decensored and for bad, BAD things to happen
I am impressed to see already couple of providers serving Kimi-K3 on openrouter: https://openrouter.ai/moonshotai/kimi-k3
the page is now throwing a 404 11 minutes out.
Did the link change or anything?

Getting 404 Sorry, we can't find the page you are looking for.

Attention replaced recurrence over tokens in 2017, this does the same over depth of the layers. It's apparently not an entirely new idea, but also an elegant reapplication of the attention mechanism.
From a cursory glance on huggingface, the files don't add up to 2+TB. Unless it adds up to that when you extract the multiple ~17GB files, if that's the case then that's some crazy compression.
Kimi K3's full weight is now live on Hugging Face https://huggingface.co/moonshotai/Kimi-K3
I am going to create a uncensored version of this one. Looking forward to design my own meth lab at home. Just kidding. But also not kidding. I like uncensored versions
HF says activation function is "SiTU-GLU", but I can't find any info on that?

Anyone know / is this a typo?

Even the python code for inference seems to use normal activations.

is there a realistic way to distill 2 consumer hardware friendly models with max ~200B and ~20B? Qwen did it, but would it be possible for 3rd parties (unsloth etc)?
They release it before I could even get a kimi subscription because of waitlist? lol hard to believe that I might get a kimi subscription from a third party