For fast inference, the HuggingFace cofounder, Thom Wolf recommends their text-generation-inference library https://github.com/huggingface/text-generation-inference

https://twitter.com/Thom_Wolf/status/1664613366751526914