back
user profile
apitman
13,487karma·3,567submissions·December 11, 2014
about
Software architect on the iobio team at the University of Utah Eccles Institute of Human Genetics. I'm passionate about the application of computer science to solving health problems. Hans Rosling is my hero.
Personal site:
apitman.com
Projects:
IndieBits.io - A community for data ownership, self-hosting, and decentralization.
LastLogin.net - A free, privacy-focused login provider
TakingNames.io - Domain names for self-hosters
boringproxy.io - Simple, e2ee tunneling proxy
droplock.apitman.com - Simple secure secret sharing
recent activity (3,567 total)
comment
Currently software is king when it comes to inference performance though.
comment
You should be getting way more than that on a 6000 pro even today. I'm getting 40tok/s on a pair of 3060s. You can ask a SOTA model to optimize your setup for you.
comment
Aren't things like KV size inherent to the model?
comment
I wonder how this would stack up against 4x RTX 3060, assuming you have the physical room for them.
comment
Interesting. Why don't the unsloth guides ( https://unsloth.ai/docs/models/qwen3.8 ) mention this? Do they already include the fixes in their GGUFs?…
comment
Check out OpenCode Go as well. They give some Kimi K3, Qwen3.8 Max, and GLM5.2 (probably 5.3 soon?) usage which may cover your needs for $10/mo
comment
Yeah make sure you're using MTP and potentially tensor parallelism.
comment
This is specifically about unsloth having day-1 GGUF's available.
comment
I used GPT-5.6 Sol high to optimize it, and it claimed it was getting 50. I'm seeing ~40 on my goto smoketest: "Make me a vector add in CUDA". Funny side note. It successfully one shot …
comment
Running it on 2x3060 now. Works pretty well but VRAM is tight . 4bit quants. 1x128k context, 8bit KV, MTP on.
comment
The Luna price dropped the day before a massive update to Flash (0731 update). GP may be referring to that version.
comment
Look at OpenRouter. They have lots of providers
comment
But why? Luna Max is almost the same intelligence as Terra xhigh and way way cheaper. And Terra max is almost the same as Sol high. I just don't really see a place for Terra but slower.
comment
Wait people use terra?
comment
Are there any projects that track how much usage of each model translates to how much percentage drop in weekly/5hr windows?
comment
https://artificialanalysis.ai/models/grok-4-6
comment
This is what I do. Nice benefit is it lets me passthrough my GPU and share it between multiple containers. Incus is awesome.
comment
Fun fact: QEMU runs natively on windows and supports acceleration with WHP. It works surprisingly well.
comment
> OpenCode Go currently offers $120 for $10 on DeepSeek Flash v4 At DeepSeek's absurdly low rates or market rates?
comment
I'm getting like 25 tok/s on 2x RTX Pro 6000. This is with llama.cpp, but I had GPT tune it for me. I was under the impression vLLM was at most ~2x faster, and usually for highly parallel lo…
comment
These are very interesting results, and honestly hard to believe, even as a big 0731 fan. If I'm reading the chart correctly, a couple observations: * deepseek-v4-flash-0731 max is better than ki…
comment
I've found it to be pretty good so far.
comment
DeepSeek has far cheaper cache pricing. That's the difference.
comment
These numbers look about right based on my experiences as well. Though for a single user I think 2x DGX Spark (~$10k) runs DSv4 Flash fairly well right?
comment
Welp. That didn't last long
comment
As low as it is, switching between providers on OpenRouter is still lower. That said, it's a fair point. For me, it boils down to things covered here: https://earendil.com/posts&#…