As LLM applications proliferate, the LLM Gateway (AI Gateway) has become a critical architectural component. The market now offers many options — OpenRouter, Moyi API, LiteLLM, Cloudflare AI Gateway, Helicone, Portkey — each with different positioning and trade-offs. How should technical decision-makers choose?
This article uses a landscape overview, deep comparison, and decision tree framework to help you find the best LLM gateway for your needs. If you self-host LiteLLM, read the section on the October 2026 supply-chain incident first.
New accounts start with a $0 balance — top up before your first request. Top up from $5. If a request returns 402 (400 on the OpenAI format) with "wallet balance is insufficient", your balance is empty — your base URL and key are fine. Top up and retry.
What is an LLM Gateway and What Does It Do?
An LLM Gateway sits between your application and LLM providers, offering these core functions:
| Feature | Description |
|---|---|
| Routing | Distribute requests to different LLM providers/models |
| Caching | Cache repeated request responses to cut costs |
| Load Balancing | Distribute requests across providers/keys |
| Monitoring | Track usage, latency, error rates |
| Cost Control | Quota management, budget alerts, usage reports |
| Security | API key management, access control, log audit |
Landscape Overview
Open Source vs SaaS
| Solution | Type | Strength | Weakness |
|---|---|---|---|
| LiteLLM | Open source SDK | Self-hostable, highly customizable | You operate it, no SLA |
| Portkey | SaaS + open source | Full enterprise features | Higher cost, complex config |
| Helicone | SaaS | Strong monitoring features | Monitoring-focused only |
General Purpose vs Claude-Specific
| Solution | Positioning | Best For |
|---|---|---|
| OpenRouter | General routing platform | Multi-model selection, comparison |
| Cloudflare AI Gateway | Edge computing gateway | Global distributed deployment |
| Moyi API | Claude-optimized gateway | Agent workflows, cost-sensitive |
6-Way Comparison
| Dimension | OpenRouter | Moyi API | LiteLLM | Cloudflare AI GW | Helicone | Portkey |
|---|---|---|---|---|---|---|
| Cache hit rate | No optimization | 99.3% | Self-host configurable | Basic cache | None | Enterprise |
| Pricing advantage | Markup | 0.3x-0.5x | Hosting cost | Usage-based | Monitoring | Enterprise |
| SLA | No public SLA | 99.98% | None (self-hosted) | 99.9% (Cloudflare) | Per plan | Enterprise SLA |
| Agent optimization | None | Deep | Self-configure | None | None | Limited |
| China-friendly | Poor | Good (Alipay+CN) | Self-configure | Okay | Okay | Poor |
| Deployment | SaaS | SaaS | Self-hosted | SaaS | SaaS | SaaS+self-hosted |
| Best for | Individual/small team | Individual/small team | Technical teams | Enterprise | Monitoring teams | Enterprise |
Decision Tree
What do you need?
│
├─ Price-sensitive, mainly Claude Code / Codex
│ └─ Moyi API (best cache + discounts)
│
├─ Need 200+ model catalog
│ └─ OpenRouter (most models)
│
├─ Need self-hosting, high data privacy
│ └─ LiteLLM (open source, self-host)
│
├─ Need global edge node acceleration
│ └─ Cloudflare AI Gateway (edge computing)
│
├─ Mostly monitoring and observability
│ └─ Helicone (strongest monitoring)
│
└─ Enterprise-grade solution with support
└─ Portkey (enterprise complete)
Moyi API's Differentiation
Moyi API's positioning is clear among the six solutions:
Why Claude Code / Codex users are the best fit:
- 99.3% cache hit rate: 80%+ of agent token consumption comes from repeated context. Moyi API's caching is designed for this, reducing effective cost to 10%-30% of official pricing.
- Request shaping reduces ban risk: Built-in request shaping, stable IP pool, compliant call patterns — solves the Claude Code ban anxiety.
- First $25 at 50% off: Minimal trial cost, ideal for individual developers and indie hackers.
- China-friendly: Alipay, Chinese docs and support, APAC low-latency nodes.
- 99.98% SLA: For automated agent workflows, this means your agents won't fail while you sleep.
The October 2026 LiteLLM Supply-Chain Incident: What to Do If You Were Affected
If you self-host LiteLLM, check this before anything else. On October 21, 2026, LiteLLM's release pipeline published a compromised litellm==1.107.0 containing malware that steals credentials (LiteLLM's incident post):
- PyPI:
litellm==1.107.0was live from 11:39 to 13:23 UTC, about two hours. - Docker Hub: the
v1.107.0,latest,main-latestandmain-stabletags pointed to the compromised image from 11:46 to 14:20 UTC, about two and a half hours. GHCR served the same tags until 13:51 UTC. - Not affected: earlier versions, and enterprise customers pulling from LiteLLM's private registry.
Root cause. In LiteLLM's root-cause post of October 23, the team says an administrative key to an internal LiteLLM Agent Platform deployment was committed to a public repository on May 27, 2026. The file was removed, but the key was never revoked. On October 10 the attacker used it to get owner-level GitHub access, then built and published the malicious release. The Register and The Record covered the root cause on October 26.
Are you affected? Yes, if you installed an unpinned litellm with pip during the PyPI window, or started one of the Docker tags above during the Docker window. Run pip show litellm, and look for litellm-1.107.0.dist-info, cook.pth and sitecustomize.py in site-packages. Finding one confirms exposure; not finding one does not prove the environment is clean.
What to do, per LiteLLM's incident post and Datadog Security Labs:
- Don't start with
pip install: any Python start in the affected environment runs the malicious files again. From the shell, deletecook.pth,cook.py,sitecustomize.pyand any matching.pycfiles from every affected site-packages directory, and kill the leftover Python process. - Reinstall pinned to
litellm==1.106.2, or better, rebuild the images and virtual environments from scratch. - If the host had outbound internet access, assume the credentials were stolen. From a clean machine, rotate everything the host could reach: every provider API key stored in the LiteLLM proxy database, the LiteLLM master and virtual keys, the
DATABASE_URLpassword, cloud IAM role credentials and any other secret in the host's environment. - Check your logs, such as cloud audit logs and your providers' usage dashboards, for use of those credentials from outside your infrastructure during and after the exposure window.
Microsoft's analysis covers how the startup hooks steal credentials, if you need detail for an investigation.
Self-hosting LiteLLM is still a reasonable choice if you need data to stay inside your own infrastructure, want full control over routing logic, or already run the ops to pin versions, verify signed images and rotate secrets. LiteLLM signs its Docker images and rotated its signing key after the incident. If you would rather not run and patch a proxy yourself, a hosted gateway means you only manage one API key; see the migration notes below.
Migration Paths
OpenRouter to Moyi API
- Migration cost: Near zero (OpenAI-compatible API)
- Benefit: 70-90% cost reduction, caching, request shaping
LiteLLM to Moyi API
- Migration cost: Low (self-hosted to SaaS)
- Benefit: Eliminate ops overhead, get SLA and support
Cloudflare AI Gateway to Moyi API
- Migration cost: Medium (API differences)
- Benefit: Agent-workflow optimization
FAQ
What's an LLM gateway vs an ordinary API relay station?
An LLM gateway provides routing, caching, load balancing, monitoring, and security — enterprise-grade capabilities. An ordinary relay station is simple API forwarding. Moyi API is an LLM gateway with full capabilities.
I'm an individual developer. Do I need an LLM gateway?
If you mainly use Claude Code / Codex and are cost-sensitive, use a SaaS gateway like Moyi API directly. Skip the self-build overhead.
How should enterprises choose an LLM gateway?
Evaluate: SLA guarantees, data privacy (self-hosting or private deployment support), compliance, ops overhead, team technical capability. Start with SaaS, then decide on build vs buy based on scale.