Market Landscape
Last updated: 2026-03-09 — refreshed during daily scans when significant changes are detected.
Overview
The AI gateway and LLM routing market is segmented by approach: infrastructure-layer (Cloudflare, Vercel), platform/governance-layer (Portkey), open-source self-hosted (LiteLLM), and marketplace-first (OpenRouter). Helicone (observability-focused) was acquired by Mintlify in March 2026 and is winding down as an independent competitor.
Competitive Matrix
| Capability | OpenRouter | Cloudflare | Portkey | LiteLLM | Vercel | Helicone |
|---|---|---|---|---|---|---|
| Models | 300+ | ~6 providers | 250+ | 100+ | Growing marketplace | 100+ |
| Unified billing | Yes | Yes (2026) | Yes | No (BYOK) | Yes (zero markup) | Yes (0% markup) |
| Caching | No | Edge caching | Yes | Yes | No | Yes |
| Guardrails | No | No | 50+ | Enterprise only | No | No |
| Observability | No | Basic analytics | Full | Full | No | Full (winding down) |
| RBAC/Governance | No | Via Cloudflare | Full | Enterprise | No | Team tier |
| Compliance | None | Enterprise | ISO/SOC2/HIPAA/GDPR | Enterprise | SOC2/PCI/ISO | SOC2/HIPAA |
| Self-hosted | No | No | Yes (OSS) | Yes (OSS) | No | Yes (OSS) |
| MCP/Agents | No | Agents SDK | MCP Gateway | MCP sessions | AI SDK | No |
| Pricing model | Markup | Usage-based | Usage-based | Open-core | Zero markup | Subscription |
Direct Competitors
Tier 1 — High Overlap (7/10)
Cloudflare AI Gateway — Infrastructure proxy with caching, rate limiting, and analytics for Cloudflare customers. New unified billing (2026) lets customers pay for third-party AI on their Cloudflare invoice. Threat is distribution: millions of existing Cloudflare customers get AI Gateway bundled. Limited provider coverage (~6 providers) keeps overlap moderate for now.
Portkey — Most feature-complete AI gateway. 250+ models, 50+ guardrails, full governance stack, compliance certifications. Processing 25M+ requests/day. MCP Gateway positions for agentic future. Strongest in enterprise segment where OpenRouter is weakest. Key risk: if a team needs both routing and governance, Portkey does both.
Vercel AI Gateway — AI model access integrated into frontend deployment platform. Zero markup on tokens with $5/mo free credits. AI SDK (open-source TypeScript toolkit) drives adoption. Expanding AI integrations marketplace (xAI, Groq, DeepInfra, Perplexity, etc.). Captures TypeScript/Next.js developers but weak for backend/agent workloads.
Tier 2 — Moderate Overlap (6/10)
LiteLLM — Most direct open-source routing competitor. Unified OpenAI-compatible API across 100+ providers. Self-hosted for teams needing on-prem control. Very active development (40+ contributors, releases multiple times per week). Requires DevOps investment (2-4 weeks setup) which limits adoption to technical teams. Lists OpenRouter as a supported provider, making it partially complementary.
Tier 3 — Declining (4/10)
Helicone — Acquired by Mintlify (March 3, 2026). Entering maintenance mode. Processed 14.2 trillion tokens across 16,000 organizations. Team joining Mintlify to build "next generation of knowledge infrastructure." Open-source codebase remains for community forks. Their 16K displaced orgs represent an acquisition opportunity.
Adjacent Players
| Player | Category | Potential Threat |
|---|---|---|
| AWS Bedrock | Cloud provider multi-model access | High — enterprise distribution, but vendor lock-in limits appeal |
| Azure AI Studio | Cloud provider multi-model platform | High — Microsoft/OpenAI relationship, enterprise features |
| Google Vertex AI | Cloud provider ML platform | High — Gemini models, enterprise integration |
| Langfuse | Open-source LLM observability | Low — could add gateway features but focused on tracing/evaluation |
| Braintrust | AI evaluation and prompt management | Low — complementary tooling, not a gateway |
| Weights & Biases | ML experiment tracking expanding to LLMs | Low — monitoring focus, unlikely to build routing |
Potential Entrants
- Stripe — Payment infrastructure could naturally extend to AI API billing/routing. Strong billing relationships with developers. Would be a serious threat if they entered.
- Datadog — Observability leader adding AI monitoring. Could extend to gateway/routing as a natural upsell.
- Netlify — Following Vercel's lead, could add AI gateway to compete in frontend AI development.
- Supabase — Backend-as-a-service with AI integrations. Could add model routing alongside existing database/auth services.
Market Trends
Consolidation
Helicone's acquisition signals that standalone AI observability may not sustain independent businesses. Expect more M&A as the market matures. Gateway features are being absorbed into larger platforms.
Enterprise Governance
Growing demand for guardrails, compliance, and audit trails as AI moves into regulated industries. Portkey and LiteLLM are best positioned. This is the most critical gap for OpenRouter's enterprise expansion.
Unified Billing Wars
Cloudflare, Vercel, and Helicone all now offer consolidated AI billing. The market is converging on single-invoice convenience. Vercel's zero-markup approach puts pressure on margin-based models.
MCP and Agentic AI
Model Context Protocol is becoming a standard for AI tool use. Portkey's MCP Gateway and Cloudflare's Agents SDK are early infrastructure plays. As agents become mainstream, gateway needs shift toward longer-running requests, state management, and multi-step orchestration.
Open-Source Gateways
LiteLLM and Portkey's OSS options create strong self-host alternatives. Open-core models (free OSS + paid enterprise) are proving effective for developer adoption. OpenRouter's managed-only approach is a differentiator (zero ops) but also a limitation (no self-host option for regulated industries).
Provider Proliferation
New model providers are launching weekly. Gateways that can quickly add providers have an advantage. OpenRouter's 300+ models lead here, but Portkey (250+) and LiteLLM (100+) are catching up. Vercel's marketplace approach (native integrations) could scale quickly.