#python #ai_gateway #anthropic #azure_openai #bedrock #gateway #langchain #llm #llm_gateway #llmops #openai #openai_proxy #vertex_ai

LiteLLM is a tool that helps you use different AI models from various providers like OpenAI, Azure, and Huggingface in a simple way. Here’s how it benefits you You can call any AI model using a consistent format, making it easier to switch between different providers.
- **Consistent Output** You can set budgets and rate limits for your projects, helping you manage costs and usage efficiently.
- **Retry and Fallback Logic** It supports streaming responses and asynchronous calls, which can improve performance.
- **Logging and Observability**: You can easily log data to various tools like Lunary, Langfuse, and Slack, helping you monitor and analyze your AI usage.

Overall, LiteLLM simplifies working with multiple AI providers, makes your code cleaner, and helps you manage resources better.

https://github.com/BerriAI/litellm

#rust #ai_gateway #ai_gateway_support #envoy #envoyproxy #gateway #generative_ai #llm_gateway #llm_inference #llm_proxy #llm_routing #llmops #llms #openai #prompt #proxy #proxy_server #routing

Plano is an AI-native proxy server that handles key tasks for agentic apps like routing between agents, smart LLM model selection, safety guardrails, and automatic traces for observability. Define agents in simple YAML, write basic HTTP code in any language, and start Plano to run multi-agent systems without custom plumbing or framework lock-in. You benefit by building and shipping reliable agents to production much faster, focusing on core logic while gaining safety, low latency, and easy scaling.

https://github.com/katanemo/plano