Excited to try this once y'all get self-hosting! Super useful that you'll also support structured output and openai style functions for models where it's not supported by the native API (that's been a big painpoint for breaking out of openai).
back
2 comments
We're really excited for our self-hosted solution too! Adds another set of challenges to the mix for us, but we definitely see the value.
We also didn't think that our structured output support would be getting much buzz, but seems like there's some demand for a good solution there so we'll continue to invest in it.
We're using Maitai's structured output in prod (Benchify, YC S24) and it's awesome. OpenAI interface for all the models. Super consistent. And they've fixed bugs around escaping characters that OpenAI didn't fix yet.