back
782 comments
Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it.

Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks.

The VRAM usage seems way less efficient than Gemma 4 or Glimmer though, with 32K of context taking 2.5GB of VRAM. With those, even with MTP or a DFlash model loaded, you could still fit 256k-768k of context. With Qwen 3.8 27B I can't even fit 128k if I quantize V to Q4_0. Maybe with some trial and error I can find some settings that perform well enough with a larger context window that it's still useful for longer tasks.

Lots more testing to do, though I was getting some decent results out of Muse Glimmer which was more than twice as fast and supported huge context windows, managing to solve some bugs that Gemma 4 struggled with. I can't even begin to throw that task at Qwen, because just the prompt alone would use the entire context window and then it would reason for probably that same amount.

If you've got a 32GB card, it should be a decent model even if it really is memory hungry.

EDIT: Tried a few kv cache quantization settings, but it failed with those. I designed this benchmark to be pretty brutal in the face of KLD and any reasoning quality loss, so it's not too surprising. Gemma 4's QAT held up pretty well, at least and could consistently complete it.

Can you please tell which Gemma 4 variant managed to correctly reason through your private benchmarks? Was is Gemma 4 31B?

What quantizations and context lengths did you use for Gemma 4 and Qwen 3.8 27B?

I am asking because I can't even load Gemma 4 31B on my GPU with any reasonable quantization (even with small context), while I can run Qwen 3.8 27B with large context and good quantization...

"correctly reason through one of my private benchmarks"

i would also like to make one myself for my testing. could you give a rough idea or an outline or point in the general direction on what to do?

Qwen’s 3.6/3.8 27b actually has some algorithmic advantage when it comes to the kv cache size needed, so it actually needs less memory at equivalent context. My experience using both in vllm supports, with considerably more overhead in context size on these models than Gemma 4 31b, and better performance in most tasks I’ve tried on both models.
In my experience MTP's speed increase doesn't seem to justify the apparent loss of success at the edge, it would have to be at least 4x faster to meaningfully churn through the first 3 failures in the time it would have taken to do it once without

What exactly are you doing that the prompt is eating an entire 65536 window? Surely it would be better to let it use any number of approaches that call tools to access that in parts and reason/summarize to an output file as it works through the whole thing. This allows it to keep the initial instructions in the start of the window and toss out the middle as it goes. IME many people who have written off local models entirely are, for lack of a better term "not holding them right" and consider them worthless.

Definitely also try IQ4_NL for K/V if you haven't. Because it's non linear it's far more hit/miss from model to model and especially quant to quant, I've found generally that it shines brightest when you start with a Q6K+ quant that you otherwise wouldn't bother with because of its size, which it then makes up for in both inference speed and often a larger context.

I do agree about Glimmer, though. It is quite good, far better than the benchmarks let on, especially in heavily agentic cases where it needs to rampage around the OS and utilize many different utilities to zero in on things. It is especially good at being told to try something itself, and if/when it fails, try Qwen, and if Qwen can't do it, call out to Deepseek.

I was quite impressed by Muse Glimmer, and while I am sure people will observe that it is less good on benchmarks, my first experiences with this new 27B have been somewhat exasperating, whereas testing Muse Glimmer was rather fun. I have not tested either in an agentic context, mind you.
> The VRAM usage seems way less efficient than Gemma 4 or Glimmer though

Maybe it's implicit that you're using llama.cpp (although you don't mention GGUF), but it's hard to reach concrete conclusions about the model architecture based on one implementation in one runtime.

Have you tried Muse 30B yet? I have been impressed with it. I have Qwen 3.8 27B hammering away right now against Muse. And Muse is doing a little bit better.
> Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks.

I don't expect you to blab publicly about your private benchmark, but what sorts of reasoning does it require?

> correctly reason through one of my private benchmarks

Want to say more about these private benchmarks? :)

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare.

(No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a small chainring.")

I ran that on an M5 Max MacBook Pro using LM Studio and their 17GB GGUF: https://lmstudio.ai/models/qwen3.8

It took 21 minutes(!) and used 22,276 reasoning tokens to produce 3,223 tokens of output.

(For the "they're training on your benchmark now" crowd, all of that cheating didn't prevent it from spending 20 minutes thinking about the task first! You can see the reasoning trace in the link I shared.)

For comparison, here's one I got from qwen3.8-2.4t-a95b on OpenRouter, which is pleasingly animated: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, write a todo list app skeleton that works, I can work with it.

SWE: model is strong for its size. It one-shotted the Web app, had no bug. The Rust rewrite only had one bug (reordering didn't work immediately - fixed in one prompt). Committed locally then pushed to my GitHub (https://github.com/DexterLagan/RusTODO). Can't complain. If it can do that reliably, I can use it to make whatever I need.

General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well.

Performance on an admittedly overpowered laptop: 15 tokens/s in power save mode on this MacBook M5 Max 48GB, and 30 in performance mode. Perfectly usable for local coding through OpenCode.

Verdict: very nice local and free backup to my usual GPT/Claude/DeepSeek for code. Good for Web searches via Brave search tool calls. What more do you want from a small local model?

There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for".

"Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman.

I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which seem to perform poorly.

Other notes: it uses the trick of repeating the prompt in the thinking trace.

It also worries about hidden chain of thought appearing in the final answer. It talks about "desired oververbosity 9", which is new. A bit GPT-ish.

It is being extraordinarily thorough in thinking through one of my code requests, but I don't know if the net result will be any better than the 35B MoE.

I asked it to ask me clarifying questions — it did, and it offered me a list of defaults I could simply agree to.

I don't think it is necessarily overthinking in the looping sense, but it is in the being exhaustive sense. I need to explore how it does with a tighter reasoning budget.

I am impressed but I am definitely in Camp Please-35B-A3B-When? here, because on an M1 Max this isn't really practical. I hope they do one, though I think they may not.

As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc.

[0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

If anyone else is running this on an RTX 5090, https://github.com/Neroued/ninfer as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.
One thing a lot of people don't seem to factor when hyping Qwen is how much models like this tend to 'overthink' with seemingly endless 'second guessing'. 3.8 seems no different from what I've tried thus far.

As capable as it is, it's hard to justify using it when a competing model (e.g. Gemma4:26b-a3b) can consistently achieve the same or similar response with only 1/10th as many 'thinking' tokens, achieve much higher tokens/second, and take a small fraction of the time. I suppose 'YMMV' depending on your use case.

Also, I haven't used it enough yet to see if it's prone to infinite looping, but its predecessors sure were.

If the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it.

I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model.

But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as Opus 4.6 was and in many cases far better.

Thank you Qwen team for this release.

Compared to closed weight (especially unreleased and access-limited) and open weight/source large sparse MoE LLMs/VLMs, open weight/source small dense models benefits public the most because they just reaches more people.

Compared to Qwen 3.6, 3.8's thinking style changed drastically. With xhigh budget, it thinks a lot MORE, and longer thinking session directly translates to better performance. This tradeoff between performance and computation, memory, etc. is meaningful to me.

However, because Qwen 3.6 and 3.8 share the same architecture, with 32GB vram, llama.cpp, IQ4_XS model, MTP and FP16 mmproj, I can only get 200k context, which is not good compared to 640k context of muse glimmer. Hopefully this problem will be solved in Qwen 4.0 release.

Wow. This model is so good, and we have GLM 5.3 (seems great voor security related work) and Deepseek. In a few months we'll have Fable/Sol-like capabilities that are not coming from the big US companies. I feel as a programmer that that is more than enough. How wil OpenAI and Anthropic survive when frontier model intelligence becomes commoditized?
Image->html test for this.

Original images: https://image.non.io/neonRamenDesigns.webp

Qwen 3.8 build: https://html.non.io/neonRamenQwen3.8-27b

Overall I'm very impressed with how well this did. It's a big improvement over 3.6, and it feels on-par with some much, much larger models. I think this one is on-par with Gemini 3.7 Flash.

One thing to note - the build for this on my RTX 6000 pro blackwell took a long time. Easily one of the longest builds I've done. It took around 2 hours to build the site. Obviously we'll have some quants for this soon that will accelerate things, but I was still surprised with how long it took.

Comparison builds from this week:

https://html.non.io/neonRamenGemini3.7

https://html.non.io/neonRamenGLM5.3 (note: non-multimodal)

I hope really badly that we'll get a new 35B A3B or similar MoE model!

I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model <100B would be the sweet spot (when you have the VRAM but not the TDP or compute power). Heck, I'd gladly take A5B or A8B or even A10B as a sort of middle ground.

Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick.

I care a lot about speed and efficiency right now. For my setup I would like to have 2-3 different model families. I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting, Deepseek V4 Pro 0813 for developing, and Gemini flash lite (any recent cheap model) for repo scouting. I'll add another one in the mix for reviewing (in this case Gemini 3.7) and that's all I need.

I've tried most models except Grok.

Qwen is too expensive IMO (Alibaba Cloud subscriptions are hard to come by and I'm not spending 50 euros a month for a tool, so 18 euros it is). If it ever becomes efficient enough to run locally I will definitely look back.

Claude is slow and expensive (the cache hit prices are absurd).

OAI is pretty good, I might add it to my arsenal seeing how cheap it is.

These opinions change every day. Last week I would've never picked Deepseek until I read about the pricing. even post aug 16 it's worth it (although it's getting close to gemini pricing).

Right now my costs are 12 euros a month (z.ai) + whatever deepseek consumes. This typically isn't more than 8 euros a week. 44 euros a month and I have a setup that is doing pretty well.

I hope the bonsai team makes another 1bit quant of this model (or releases code/instructions on how to do it), using the Qwen3.6 27B on my 16GB mac mini has been wild . The 1bit quant feels like opus level… for the first couple turns. Then it has trouble eg switching from plan mode to act mode. This is mostly mitigated by starting a new session. (tbf this limitation is called out on the hf page)

I saw unsloth has 1bit quants too so I might check that out, anybody have experience with those?

So far what I am seeing in my seemingly simple "Wordpress last-login plugin" test is that in xhigh reasoning mode (the default, seemingly) it overthinks so badly that it writes terrible bushy code with edge cases caused by going down rabbit holes.

In "medium" reasoning mode, you get the classic Qwen wait/actually thinking loops you see in 3.6 that I guess will need to be interrupted in the way others do already with an over-thinking guard proxy.

(In one of my test runs it is now on "OK TRULY FINAL APPROACH" after having got through "FINAL FINAL APPROACH". Can relate)

It gets stuck in a thinking loop regarding the WordPress API and (resolvable) ambiguity in my prompt, that I guess might be resolvable with a custom skill with hints on how to look it up (and maybe with the devdocs MCP).

In Low reasoning effort mode it flies through the task and writes pretty solid code. So maybe it is me overthinking what is needed here...

We maintain German Langauge index as no one publishes or reruns these sepeartly.

Qwen 3.8 27B is a small improvement with some regressions in our benchmarks not a huge jump like benchmarks listed.

https://dach.peerbench.ai/compare?models=qwen%2Fqwen3.8-27b,...

German language has never been a big focus for asian models but they still outperform Gemma models https://dach.peerbench.ai/compare?models=openai%2FQwen%2FQwe...

So in production we have been using Gemini Flash Lite as primary and fall back to Qwen when gemini servers are overloaded or just giving us 429

I wonder how this practically compares with Muse Glimmer, especially quantized.

I've got an RX 7900 XT (20GB of VRAM) and I can run glimmer with a full 128k context window with the draft model at 65-80 tok/s.

This model, on the other hand, I get about 30 tok/s with a 30k context. Raising the context or loading the draft layers for MTP drops performance to 9-15 tok/s.

So I wonder how big the "real world" delta between Glimmer and Qwen is here. I can already run 3-bit DSv4-flash at 9-15 tok/s with 100k~ context, and I suspect it would outperform 4-bit Qwen 3.8 27B here.

I'll have to experiment and see if I just made a stupid mistake somewhere, but it looks like Glimmer might make more sense for the comically specific niche of "20GB VRAM".

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM.

llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off

Identical to the qwen3.6 config. With a prompt like "svg owl" (which can reuse quite a lot compared with creative writing or similar, so ngram-mod shines), I get about 70-80t/s like this, with a memory overclock of about 1.5GHz

The unsloth Q8kxl https://huggingface.co/unsloth/Qwen3.8-27B-GGUF for some reason is looping and going crazy on the think part (I tried to search for an email to let the guys know but didn't find one)... I used the bartowski one and that one doesn't have that issue https://huggingface.co/bartowski/Qwen3.8-27B-GGUF?show_file_...

that's using llama.cpp

llama-server \ -m ~/somePath/Qwen3.8-27B-UD-Q8_K_XL.gguf \ -np 1 --kv-unified \ -fa on --no-cache-idle-slots --reasoning-preserve \ --temp 0.2 \ --spec-type draft-mtp,ngram-mod --spec-draft-n-max 3 --spec-draft-n-min 1 \ --cache-type-k f16 --cache-type-v f16 \ --chat-template-kwargs '{"preserve_thinking": true}' \

I tried playing with all the recommended parameters from the unsloth page with no luck...

in one of the high fever ramblings it ended with amen... lol

WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build, and it built the whole thing from beginning to end.
Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch.

Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Unsloth Q4_K_M on a single 3090, llama.cpp "Generate an SVG of a pelican riding a bicycle" first try https://www.reddit.com/r/LocalLLaMA/comments/1voa3ch/comment...
Beating or comparable to Opus 4.6 in benchmarks. Opus 4.6 was released in February, 2026. So if we still want to talk about a "6 month difference" between Chinese and American AI, the sentence should now be:

Chinese (small model) AI is 6 months behind American (largest model) AI.

Any tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.
The file "Just loads" on llama.cpp, the Unsloth https://huggingface.co/unsloth/Qwen3.8-27B-GGUF is an MTP file, I see mostly the same speed on pp and generation. There has to be something wrong with those benchmarks, I find extremely hard to believe a 27B model can work similar or exceed opus 4.6.
This is one of the most important model releases since most use cases don't need SOTA/Frontier

If you want Qwen3.8-27B Serving Configs for the DGX Spark vLLM NVFP4 and RTX 4090 llama.cpp GGUF I added the setups here https://x.com/ErdalToprak/status/2088299678085308761?s=20

The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help
Big thank you to the Qwen team. 3.6 A3B was shocking good, and now I'm hoping they release an 3.8 A3B model too.

Edit: Having used qwen3.8:27b-mlx on MBP M4 64GB, I get around ~45 tok/s. A3B would be great for smaller devices, but it's definitely usable. As I understand it it's a mixture of MLX and MTP.

If the benchmarks are a real indication, we now have a local model that is runnable on a high-end personal PC that trades blows with the leading model Claude Opus 4.6 Max from half a year ago.

Insane if that is the case. Downloading now!

Qwen3.6-27B has been the main LLM powering my little agentic stack. I have adopted the test and verify approach to any models allowed to run on my machine. When the "heretic" version drops, I will fire up the harness and test. Super excited to see how it stacks up against Qwen3.6!!!
27b dense model at Opus 4.6 level

Opus at home

I hope there also will be a new ~10b variant

Yeah this is the first model I have been able to run locally that actually feels useful, this is unreal I am considering cancelling my claude sub and going to just api (maybe GLM?) for really hard tasks.
On my RTX3060 - 12GB VRAM + 24GB RAM , with below command

       ollama run qwen3.8:27b --verbose "explain mmap”, 

 I got  2.41 Tokens/s, Not sure if that can be improved considering VRAM doesn’t fit the entire, model.
Additional details:

total duration: 8m18.2870918s load duration: 612.105ms prompt eval count: 12 token(s) prompt eval duration: 2.900965s prompt eval rate: 4.14 tokens/s eval count: 1193 token(s) eval duration: 8m14.660618s eval rate: 2.41 tokens/s

System spec: NVIDIA GeForce RTX3060 AMD Ryzen 5 1600 Six-Core B450 AORUS M Mother board. NVIDIA-SMI 620.02 Driver:620.02, CUDA Version: 13.2

I am so happy right now, qwen3.6-27b was an absolute game changer. To see another one in the same league.. phew
I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...
It's an amazing model, it's GLM-5.2 level[0], running locally...

I tested it on my 3090, took like 8 hours to benchmark it and my room became a furnace (35+ deg outside temp), but it's really good.

Now, in theory, you can talk directly to your computer and tell it what to do, and it does everything locally.

[0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-medium/z-ai-gl...

These are massive improvements - and something you can actually run on a laptop.
vllm on 4x 5090 is getting ~20 tok/s with mtp on (their own thread on the hf card). i had qwen3.8-27b up the day after release, one rtx pro 6000, 140 tok/s spec, 0.156s first token, full 262k. image and video on the same api. numbers: https://github.com/avifenesh/memra try it: https://inference.tiyuvta.ai/app $0.38 in / $0.20 cache / $2.60 out. openrouter's only host right now is 23 tok/s at $0.45 / $3.20.