I use Gemma 4 12B in the 4-bit quantization for all sorts of vision tasks (image sorting, classification, description). It's also good for the same sorts of things for text (but there are probably better/faster models for text, 12B just happens to excel at vision tasks). The Qwen 9B is also very good for those tasks. If you need to do any kind of "search the web, grab some data, do some kind of action" tasks, these small models are perfect for that. Scraping data in a fuzzy format into a database or report or spreadsheet, producing a dashboard of news, etc.

Small local models can also be used for sub-agent tasks in most agent harnesses. But I'd probably run a larger MoE for that; they're faster and have broader knowledge. The dense models, even very small ones, are not blazing fast.

I don't code with any models small enough to run locally, at least not so far. Qwen 3.8 27B might be the tipping point, though. It's looking really promising, though it's probably slow enough that I won't ever actually use it. I'd rather pay $100/month for a faster model, even if Qwen 3.8 turns out to be smart enough for most of my work. Running it locally with the 8-bit quantization is going at 12-30 t/s, depending on how much context it's chewing on. So, if all you do with AI is coding, then you're better off doing it in the cloud.

But, there's lots of things a small model can do that aren't coding.

they are all overlapping but:

categorization, information retrieval, semantic search, image description

also with the model as part of an agentic system with tool calling

(edit: it is quite impressive what a small model in a feedback loop can do)

9B Qwen models are good and fast for local python coding tasks.