agentgateway

Overview

agentgateway is an open-source HTTP and gRPC gateway for teams that need one control point for service, model, MCP, and agent-to-agent traffic. It suits platform engineers, tool-builders, and technically confident AI teams running infrastructure themselves. The broad policy and telemetry toolkit is a strong fit for governed deployments, while organizations wanting a fully managed endpoint should choose another approach.

Key features

The LLM gateway exposes an OpenAI-compatible API while connecting to OpenAI, Anthropic, Bedrock, Gemini, Vertex, and open-source Llama deployments. That compatibility can simplify provider switching for applications, and the extensive integration list also includes Azure OpenAI, Mistral, DeepSeek, Ollama, xAI, and Snowflake. For self-hosted inference, routing to vLLM, TGI, and Triton can account for latency, cost, and model characteristics, making it useful when a platform mixes cloud APIs with its own GPU-backed services.

MCP capabilities go beyond proxying: server discovery, scoped tool calls, and audit trails provide a basis for controlling which agents can invoke which tools. A2A routing links runtimes such as LangChain, CrewAI, and ADK. These features are valuable for multi-agent platforms, but they require teams to design scopes and review recorded activity rather than assuming the gateway supplies a finished governance policy.

Security controls include JWT and OIDC, API keys, authorization policies, TLS and mTLS with certificate rotation, and rate limiting. MCP guardrails inspect calls and responses for prompt injection and unsafe payloads; prompt guards add checking and redaction. Together they address identity, transport, abuse, and agent-specific risks, although operating the policies and certificates remains the customer's responsibility.

Observability is built around OpenTelemetry, with Prometheus token-usage histograms, Jaeger traces, and request logs that include latency, token counts, and realized USD cost. Scoped virtual keys plus per-key or per-team token and dollar caps help enforce budgets, while dashboards group spend by model, provider, and user. This combination is particularly practical for platform teams accountable for both reliability and AI usage bills.

Pricing

The free plan has no subscription charge and is self-hosted, so its trade-off is operational rather than a published seat or quota limit: your team supplies deployment, upgrades, infrastructure, and policy administration. TLS automation, canary routing, rate limiting, and authorization policies are included capabilities. Commercial help is available through partners; Solo.io's enterprise offer includes 24×7 coverage, upstream collaboration, and best-practice guidance. Choose the free plan when you have the platform skills and control requirements to run the gateway; budget for partner support when downtime or specialist guidance carries a higher cost.

Platforms

agentgateway targets API and Linux environments and is self-hosted. Deployment is offered as a standalone binary, Docker container, or Kubernetes installation. It is therefore a natural fit for infrastructure already managed through Linux, containers, or clusters, but not for readers seeking a native desktop client or a hosted web service.

Who it's for

Platform teams can use the gateway to standardize access across model providers, internal inference, MCP tools, and agent runtimes while applying shared authentication, guardrails, telemetry, and spend limits. Tool-builders benefit from the OpenAI-compatible interface and scoped tool access. AI enthusiasts can join the community and run the open-source project, provided they are comfortable maintaining self-hosted software. Small application teams without operations capacity may find the deployment burden outweighs the breadth of controls.

Pros and cons

Pros

• Wide traffic coverage: One gateway handles HTTP, gRPC, LLM, MCP, and A2A flows instead of forcing separate control layers.
• Provider and inference flexibility: OpenAI-compatible access and routing for both hosted providers and vLLM, TGI, and Triton support mixed architectures.
• Governance with measurable spend: Identity controls, guardrails, audit trails, token and dollar caps, and cost-focused telemetry give platform owners concrete levers.
• Deployment choice: Binary, Docker, and Kubernetes options accommodate different self-hosted operating models.

Cons

• Self-hosting is mandatory: Teams must operate infrastructure, certificates, upgrades, and policies rather than handing those tasks to a vendor.
• Controls demand administration: Guardrails and authorization help only when teams define appropriate rules and monitor their results.
• Support is partner-based: Organizations needing guaranteed assistance must arrange a commercial support relationship.

Alternatives

Choose ngrok instead when you want a freemium service with web and desktop platform coverage, online endpoints, and a free allowance of up to three endpoints and 1GB of transfer.

Kusk Gateway is a simpler open-source, MIT-licensed gateway for teams prioritizing a conventional API gateway over agent and model governance.

Pick BFE Ingress Controller for an Apache 2.0-licensed ingress option, or Traefik Proxy for an open-source ingress controller, reverse proxy, load balancer, and service auto-discovery.

HAProxy Ingress fits Kubernetes-focused deployments that can meet its cluster requirement. Airlock Microgateway targets small environments or local development with a throughput-enforced, six-month community license.

Contour is an open-source Kubernetes ingress controller with community support, while Envoy Gateway is an open-source Apache-2.0 project for teams seeking that ecosystem.

Verdict

Choose agentgateway if your platform team needs centralized routing, security, guardrails, observability, and cost controls across models, tools, and agents, and can run a self-hosted gateway. Look elsewhere if you need a managed service or do not have the operational capacity to maintain infrastructure and policies; its breadth is most valuable when you can actively govern it.