OpenZiti LLM Gateway
6.0#27 in llm gateway softwareAPILinuxmacOSSelf-hosted
Visit| Fact | OpenZiti LLM Gateway | TrueFoundry AI Gateway |
|---|---|---|
| Deployment | The project describes deployment as one Go binary and one YAML file, with no database, message queue, or sidecar required. | The gateway supports SaaS and, for Enterprise, VPC, on-premises and air-gapped deployment options. |
| Headquarters | Charlotte, North Carolina, United States | San Francisco, California, United States |
| Observability | When enabled, Prometheus-exported metrics include request counts, latency, token counts, routing decisions, provider errors, and endpoint health. | Monitoring covers requests, tokens, cost, latency percentiles, time to first token, inter-token latency and request-level traces. |
| Purpose | It is an OpenAI-compatible API proxy that routes requests to OpenAI, Anthropic, and compatible backends such as Ollama and vLLM. | TrueFoundry AI Gateway sits between applications and model providers to manage AI model access, routing, governance, observability, security and cost controls. |
| Routing | Optional semantic routing uses keyword heuristics, embedding similarity, and an LLM classifier to choose a model when the request omits one. | The gateway supports latency-based routing, weighted load balancing and automatic fallbacks to secondary models. |
No prices published.
It suits teams needing a self-hosted gateway for model routing, provider translation, fallback, usage analytics, budget controls, or virtual keys. The project specifically addresses gateways, GPU servers behind NAT, and clients needing access without opening ports.
It suits teams that need a shared gateway to manage model access, routing, usage, and governance. Enterprise deployment options may suit organizations that need VPC, on-premises, or air-gapped hosting.