back
97 comments
I don't think I can handle another small model release by qwen, I'm still trying to find the limits of 3.6 27B and they are already threatening us with a new one?

But jokes aside, I love the fast iteration, these are most probably again finetunes on the 3.5 architecture that appear better in internal testing, which is still very nice to see. Putting more and more pressure on the bigger labs to perform better is always a good thing.

How good must their training pipelines be? Releasing publicly and at this rate has made them very efficient.
still waiting for a update to Qwen3-Coder-Next
Can someone explain what the current state of model benchmarking is? If you try to look up what the best locally runnable model is, you get a bunch of random blog posts using idiosyncratic criteria to rank things seemingly based on one dude's opinion.

Ideally I would love to see a leaderboard with relatively objective ranking criteria that 1. lets you filter by open weight / locally runnable, 2. filter by date of release (nothing older than x), and 3. is agnostic to hardware requirements. I just want to know what the best model is. Let me worry about how I will afford to run it.

I love the llmfit project for seeing what will run on your hardware, but it would be nice to know what I'm missing out on by not having better hardware, thus why objective hardware-agnostic ratings would be helpful.

That would be nice, but it's not going to be possible.

Any open benchmark has a very short life, since it will be pulled in and DPO / RL trained quickly for benchmaxxing purposes. So, you'll need a private test to have a hope of something fair. (These also get leaked over time, btw, so even then there's a window of usability).

These are expensive to run.

Now consider that there might be 15-20 viable quants for a given open model release; someone would have to want to pay for these private evals to be run on them. Even then, a good read through unsloth's commits and blog posts will remind you that there's quite a lot of engineering work to be done to get model inference working properly, even for models released by frontier or near-frontier labs. So, you'd want to make sure that you have a replicable 'best engineered' deployment to evaluate, or at least one that's closest to your hardware and fits the bill.

Upshot - it's much faster to download and try out a model, and possibly cheaper too. Well, cheaper since hugging face is paying the bandwidth bills.

>I just want to know what the best model is. Let me worry about how I will afford to run it.

This is a very typical manager question that I suppose many people have who fail to see the simple truth: There is no "best" model. There are only best models for certain use-cases. Sometimes you'll find these in custom community leaderboards on platforms like huggingface, but for most business applications you'll probably have to come up with your own benchmark. Most common benchmarks are pretty worthless by now because all the usual ones are being gamed hard by model providers, to the point that there are now sometimes drastic differences between models that perform very similarly on common benchmarks.

The best thing I have come up with is just make a bunch of prompts / tasks that I personally care about and need a model to know how to do. As an example, when qwen3.6 27B dropped, I ran it, kimi, claude and glm 5/5.1 on a bunch of LLM-architecture specific tasks (stuff like 'implement an incremental KV-cache for autoregressive transformer inference' or 'implement flash Attention backward pass with D-optimization') and analyze the results, who made tests, are the tests valid, does their implementation actually work or are they only claiming it to, that sort of thing.

It is a day/weekend worth of work, but I think this is the best way to determine if the model fits your need specifically. This is what lead me to finding out that qwen 27b outperformed even kimi on those tasks, and that opus tries gaslighting me when I give it a spec of something that has been proven, but no published solution exists online. All other models gave their best shot at solving it, opus just said it's not possible (even when I gave it the finished working product that obviously works).

Especially for small models (but also big ones) I think the only way to know if a model will improve your workflow is this, personal benchmarks, expanded over time, ran in private.

> Ideally I would love to see a leaderboard with relatively objective ranking criteria that 1. lets you filter by open weight / locally runnable, 2. filter by date of release (nothing older than x), and 3. is agnostic to hardware requirements. I just want to know what the best model is. Let me worry about how I will afford to run it.

Stick to artificialanalysis.ai it has become the norm

benchmarks = bs
https://xcancel.com/Alibaba_Qwen/status/2056403591464984753

> Qwen3.7 Preview lands on Arena !

> Here come Qwen3.7-Max-Preview & Qwen3.7-Plus-Preview. Alibaba now #6 lab in Text, #5 in Vision.

> Can't wait to release Qwen3.7 series models!Stay tuned! @arena

I am very interested in seeing new qwen models. Qwen3.6 27b is the first one that can do things and doesnt constantly loose "it's mind" and that can be run on a 3090 with a good context size. But it's sometimes getting into a loop.
Look on HuggingFace, there is a template that is supposed to fix the updates for the Qwen Models.

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Maybe will help you?

I've completely replaced GitHub Copilot using Sonnet 3.6 with OpenCode using Qwen3.6 27b, and it's been a great experience.
I sort of thought this about qwen3.5 35b, finally a local model that isn't a complete waste of electricity, but "upgrading" to 3.6 35b left me disappointed. It seemed more like a downgrade. But honestly I've barely used either. Subjectively they still seem far from the frontier models, but for what they can do, it's great to be able to do locally.
I had a flavor of an older version of Qwen (I forget which one to be fair) that was coding along, then lost itself in a loop, I was so confused, it was just a random greenfield "lets see how it does" type of project anyway.
Vision has become totally underappreciated, whereas I believe it brings important advantages to a model

Also, a big caveat in using Qwen models has always been its speech patterns. I do wonder how Google made the Gemma lineup so good at this

Let's hope Alibaba continues to open source its models

Agreed. Incidentally, in my testing, qwen models (qwen3.6-35b-a3b and earlier 3.5) are WAY better with vision than gemma4-26b-a4b. I would normally want to stick with gemma4 only (I use it for spam filtering), but it just doesn't cut it for vision work, and qwen models do.
I'm not much interested in vibe coding (for those who aren't aware that LLMs have other uses). The specific model I've been using with Ollama is hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:UD-Q4_K_XL and it's amazing how fast it is on 64 GB of RAM and i5-13400 CPU. No GPU on this computer. Gemma 4 E4B will think for a couple of minutes vs 3-5 seconds for Qwen. It's hard to believe how much you can do with such limited hardware using their models.
Since you are using unsloth models from HF, why not use Unsloth Studio instead of Ollama?

It is supposed to be faster + they will update the new models multiple times during the first month to correct bugs and performance issues

https://unsloth.ai/docs/new/studio#quickstart

I have a much more powerful PC and I would not call Qwen3-Coder-30B-A3B "fast" on my machine by any stretch of the word. How are you running it?
What are your use cases?
Qwen 3.6 35B (finetuned) is so good that it became standard open weights for everyday use. Is not far at all from proprietary models if you give it tools, skills and agents etc, it can actually finish the job. (Thank you Qwen team, appreciated). Using opensource now we can definitely rely to design from scratch very complicated architecture and build pretty fast the full pack. Wish to see Europe AI unleashed, wake up.
> Is not far at all from proprietary models if you give it tools, skills and agents etc,

I use Qwen 3.6 27B, the dense version of this model which is slightly better.

I don't agree that it's close at all. Maybe for some small, easy tasks, but not for working on real codebases. It's amazing for something I can run at home, but the difference between it and Opus or GPT-5.5 is huge.

It's 3.7-max; max was never open-weighted before. I don't see any smaller models in that tweet.
For coding it’s really bad. Writing is ok, chat is good. It’ll get better but it’s not that close yet
Do you have a good resource on how to finetune a model like Qwen? I am curious to try it out.
Can you share the GGUF for this specific success story? I'd like to try it for myself.
I love that open weight models are catching up so quickly. Also hilarious how far behind Grok is. I guess demand for Grok must be poor if Anthropic is able to rent resources from xAI.
Just to be clear, "Plus" and "Max" Qwen models are closed. Seems likely smaller open versions will be released, but that's not what was announced today
To play devil's advocate I do feel like Grok has a unique "feel" to it. All the Chinese models feel like GPT or Claude distillations, but Grok has a certain unique way of saying and doing things. But that said, it also feels a year behind the state of the art.
Would xAI do this if Grok was competitive?

https://www.nytimes.com/2026/04/03/business/spacex-ipo-grok-...

So glad they’re holding steady on open weights.

At least for now. Worried the Chinese team will change their mind once they have parity

Of course they will.

Right now they want to prevent the US labs from gaining any sort of self-reinforcing oligopoly on the space, and to let the ecosystem in China flourish.

That will all die sooner or later.

There I was waiting on a smaller version of Qwen 3.6 to drop so I can run it on my Mac, and then bam, they drop this.
Will they release the large models as open weight too? So far it seems only 35 or 27 B etc models are being released with nothing larger unlike before.
The jump from 3.5 to 3.6 was noticeable and set the bar. If they can keep the momentum, I’d pretty much say Qwen and China won the AI wars
Gemma 4 and Qwen 3.6 were when my local inference experiments graduated from toy challenges with much hand holding to actually full day back and forth with good ability to utilise tool calls to discover how things are glued together.

I'm not talking about greenfield dev, I'm talking about interfacing with an existing decade old codebase.

I stopped caring about benchmarks at MiniMax M2.5. I no longer want more advanced models. I want cheaper models that don't slow down when everyone else is online.
Run locally and you can now do it on an airplane
I have a tangential question. Provided that it is correct that current proprietary models are offered at below cost-covering rates (I believe this is a consensus if I'm not mistaken¹); what factor (multiplication) would have to be applied approximately to current rates to reach break even?

¹: I think I read this a couple of times but I'm not sure if correct to begin with. Can this be substantiated based on annual financial reporting or other published business metrics by OpenAI, Anthropic et al.?

Where's Grok 4.3 on the leaderboard?
There's a Grok 4.20 at #10? Maybe they just skipped version numbers for the 420 luls (are we 15 or what? wtf)
88th
lmao at opus 4.7 being a downgrade
They made it less sycophantic. Which is a good thing for mental health, but maybe a bad thing for popularity contests.
Today I learned Meta's new model is preferred to everything but claude. That is .. a real surprise! Congrats to the Meta team.