LLM Gateway

Parent: Agentic.

An LLM Gateway sits between apps/agents and model providers: routing, fallbacks, keys, spend caps, and sometimes light guardrails. Under Agentic it is the model edge — distinct from Serving (you run weights) and from Agentic Gateways (you expose agents/MCP). Without a gateway, every service invents its own provider SDK and budget leak.

Technically compare load balancing across vendors, semantic caching, request/response transforms, and how auth maps to tenants. Failure modes: a single key shared across projects, silent provider failover that changes answer quality, and gateways that become the only place logs live so your ledger never sees cost.

Members include LLM-native routers (LiteLLM, PortKey) and general API gateways used in front of model endpoints (Kong, Envoy, higress, kgateway). Prefer LLM-native when prompt/spend features matter; prefer battle-tested API gateways when you already operate them for the rest of the estate.

Feature maps such as Cost, Latency & Operational Economics, Bring Your Own Model, and Model Flexibility & Routing are the falsifiers: can you switch providers without rewriting agents, and can finance see spend per case?

Inline members for audit: LiteLLM PortKey Kong Envoy higress kgateway. Features: Cost, Latency & Operational Economics Bring Your Own Model Model Flexibility & Routing. Related terms referenced: Observability, Serving, Guardrails.

For this research catalog the acceptance test is boring and strict: after wiring the pack, can a second person replay the same scenario and get the same measurable outcome? If not, the page is still a brochure. Document the scenario, the expected number, and the log line that proves it. That habit is what separates a deepened taxonomy cell from a landscape screenshot.

Also record what you refused: tools not connected, corpora not ingested, metrics not collected. Refusals keep the cell honest when marketing pages claim universality. A taxonomy essay that only lists winners teaches less than one that names the skip conditions next to the install path.

Keep the essay technical enough that an engineer can argue install order — and short enough that a reader finishes it before opening the right-rail member list. Prefer primary docs and repo evidence over landscape blurbs when the right-rail packs disagree.

Operational discipline for this cell: name the install artifact, the health check, and the rollback. Prefer one golden path documented against a member pack in the right rail over five half-wired demos. When packs disagree, believe the primary repository README and a failing test you can re-run — not the landscape one-liner. Measure latency, cost, and error rate on a frozen scenario before declaring the subcategory “covered.” If you cannot point at a log line that proves the scenario, the essay is unfinished even when the word count clears the audit floor.

Cross-link greedily but honestly: every taxonomy wikilink should change a reader’s next click, and every pack slug should resolve to a real pack. Do not invent members to decorate the prose. Prefer fewer true links over many decorative ones. Keep Monday-shaped typing out of the runtime story; observation and ledger remain senior to any agent UI that retypes reality.