I've used LiteLLM for my 1k-user service https://LibreThinker.com Works okay-ish, but some vendors now default to MFA on API keys (eg OpenAI) which makes things tricky; I decided to simply let those fail for now.