LiteLLM

In this catalog it is the LLM Gateway option when you need to route and govern calls across model providers. Keep LiteLLM on the observe/propose side of the loop: useful signals and drafts, with human or policy authority still required before anything irreversible runs.

Applications send chat/completions to the gateway; it routes to providers, applies keys/limits, and returns responses (often with logging/fallbacks).

1. Configure upstream model providers and API credentials.
2. Set routing, budgets, and any default guard policies.
3. Send a canary completion through the gateway and inspect logs.
4. Point agent frameworks at the gateway base URL instead of raw providers.

Related: portkey · helicone · langfuse · topics/01-architecture · topics/12-central-nervous-system

• Use when: LiteLLM exposes a unified OpenAI-compatible API over 100+ model providers with routing, fallbacks, spend tracking, and load balancing (open-core) matches the bottleneck and you specifically need to route and govern calls across model providers.
• Skip when: another pack already covers the need with less ops surface, or LiteLLM would become a second place people retype truth.
• Prefer instead: compare PortKey, Helicone, LangFuse before locking the cell.

• Ops: confirm license, hosting, and release cadence for LiteLLM (BerriAI/litellm) before relying on it in a live corridor.
• Integration: bad credentials, scopes, or schema drift produce empty retrievals or runaway tool/model spend without an obvious UI alarm.
• Scope: LLM Gateway components support agents; they are not the company ledger or the approval authority.