Thanks for the kind words, we’ve improved performance now actually, follow the instructions on the core repo.
Same model should run 3x faster on the same phone.
These improvements are still being pushed to the SDKs though.
back
1 comments
Wow! 3x is huge.
I've had great experiences with gpt-oss20b on my laptop, a genuinely useful local model.
3x probably doesn't get my Pixel Pro 9 to being able to run 20b models, but its getting close!
Although GPT OSS 20B has 1.7B activated parameters which will be fast, 20B weights is a lot for developers to bundle or consumers to download. That’s the actual problem.