Would be awesome if it ran Qwen (the MoE probably won't squeeze that low, but...). This because I have hardly been able to use Gemma for any sort of useful coding.
back
2 comments
You’re right, Gemma isn’t the best model for coding (afaik more "everyday tasks" related). My first idea was to use Qwen, but its architecture was much more complex to implement in this stack. I chose Gemma so I wouldn’t spend all my time debugging custom kernels and could actually move the project forward with simpler approach
In defense of this model, Gemma is actually a very good general-purpose model that can work with multiple languages. I use it for spam classification and for processing dictation, which means that I hold the entire model in memory all of the time, which is somewhat problematic (64GB RAM total, but heavy usage by docker, databases, etc)
Gemma is a great reference model and it’s easy to work with. Once you have Gemma working well, then do the extra work to use Qwen as well.
I am using Gemma for a few tasks simply because it’s “good enough”.
I test Qwen models regularly. They are very good for English and I'm guessing Chinese, but much worse for non-English (specifically, Polish).
Gemma 4's tool calling was recently fixed; that was the main issue with agentic use in my experience.
Otherwise IMO it codes about as well as the Qwen MoE for PHP and SQL. It's a fully impressive model (though it is not as mindbendingly impressive as the 12B, which is outrageously good for its footprint)
Even the fixed version is less of a coder than Qwen.
Which model do you think currently hits the highest size-to-performance ratio for agentic tasks like tool calling?
OK so I don't have enormous agentic coding experience yet (I'm still learning about the tech more than coding with it) but:
- The consensus is that if you have a properly kitted out PC with enough VRAM, the Qwen 3.6 27B dense model is the one to beat.
Bit slow on my M1 Max so I haven't bothered with it much but I have no reason to doubt the consensus. Prism ML's new Ternary Bonsai variant of it makes it much easier to play with this model in limited RAM, but in my own toy experiments I have seen Ternary Bonsai get very stuck in thinking loops. There is another post-train variant from BottleCap called ThinkingCap, which you could try.
- The Qwen 35B MoE model is really impressive for code generation. I personally would pick this one for an older Mac or a machine with smaller VRAM; it's pretty fast, has good built-in MTP, great tool-calling.
- The Gemma 4 26B MoE has a similar capability but biased more to writing than coding. I think it writes really well, and it seems to have good knowledge of e.g. WordPress coding, SQL etc. Tool-calling let it down for agentic coding, and I haven't retested it since they fixed that
- The dense 31B Gemma 4 is large and runs slowly on my machine, but has very good general knowledge, writes well, so it should I think be better than the Qwen 27B for research tasks, and it should now be pretty solid at tool-calling.
- If you don't have much VRAM, you are not doing much coding (e.g. you want short snippets) and you want to experiment with local LLMs and perhaps in particular image analysis, the Gemma 4 12B is fun. It has an integrated vision decoder which is very impressive. Codewise, it's going to fail on long context tasks.
Thanks a lot for the recommendations!
No worries. YMMV for coding things but I've learned a lot more about LLMs this way than I think I could have from just reading about and using cloud LLMs.