Can you explain the type of quantization you support?
would https://docs.unsloth.ai/basics/kimi-k2-how-to-run-locally be faster with mirai?
back
1 comments
right now, we support AWQ but are currently working on various quantization methods in https://github.com/trymirai/lalamo