Skip to main content
Use this page to understand how ready AI Stats is for migrations from OpenRouter, Vercel AI Gateway, and LLMGateway. This review is based on the current codebase, tests, SDKs, and docs in this repo.

​
Scope reviewed

  • Public migration guides
  • OpenAI-compatible request and response compatibility
  • Batch, file, webhook, and async-job support
  • SDK and local devtools parity
  • Vercel AI SDK compatibility coverage
  • Benchmarking and migration validation tooling

​
Current parity status

CapabilityOpenRouterVercel AI GatewayLLMGatewayCurrent AI Stats status
Public migration guideYesYesYesPublic docs now include all three guides
OpenAI-compatible base URL + key swapYesYesYesCovered in migration docs and SDK examples
Model-ID verification guidanceYesYesYesCovered in migration docs via /v1/models checks
Batch + file migration readinessPartialPartialPartialGateway/API/SDK/docs now support batch create/retrieve/cancel, owned files, and result download
Routing and observability metadataStrongMediumMediumUsage UI, devtools, and batch responses expose request_id, provider, pricing_lines, and related metadata
Public benchmark toolingStrongMediumMediumInternal benchmark tooling now compares AI Stats vs OpenRouter, LLMGateway, and Vercel AI Gateway, with aggregate and live compare support when benchmark keys are configured
Vercel AI SDK compatibility proofN/AStrong across shipped modalitiesN/ALocal compatibility tests now exist for language, streaming, embeddings, image, speech, and transcription flows

​
What the repo now clearly covers

​
1) Migration docs are now in good shape

The public docs now contain platform migration guides for: These guides now follow the same basic structure:
  • inventory the current gateway boundary
  • swap endpoint and credentials
  • validate model IDs and request behavior
  • use a rollout checklist instead of a one-shot cutover

​
2) OpenRouter parity is strongest today

OpenRouter currently has the deepest repo support among the reviewed competitors:
  • a dedicated migration guide
  • an internal side-by-side benchmark path in apps/web/src/lib/internal/gatewayCompare.ts
  • OpenRouter-specific compatibility headers in the benchmark flow
  • existing product copy and internal compare UI built around the OpenRouter comparison path
This does not mean every OpenRouter feature is matched. It means AI Stats currently gives you the clearest migration and validation path for that competitor.

​
3) Batch and async support is much stronger now

Recent gateway work now gives the migration story much stronger async coverage than before:
  • batch create, retrieve, and cancel across API, docs, and SDKs
  • owned file retrieval and content download
  • webhook-aware async job visibility in usage UI
  • batch billing and terminal observability metadata
  • local gateway smoke coverage for successful and failed batch flows
That matters because async and batch flows are often where simple “OpenAI-compatible” claims stop being enough.

​
4) SDK and local tooling are stronger than the docs used to suggest

The maintained SDKs now have:
  • batch helper coverage beyond cancellation
  • file-content helper coverage
  • batch-aware devtools metadata across the maintained language SDKs
  • richer routing and pricing metadata in local recorder output
This means the migration story is no longer only “change the base URL.” The local debugging and observability experience now matters too.

​
What still needs work

​
1) Competitor comparison tooling is now mostly aligned

The internal benchmark and compare tooling now supports:
  • AI Stats
  • OpenRouter
  • LLMGateway
  • Vercel AI Gateway
That closes the earlier benchmarking gap for Vercel AI Gateway and makes the internal comparison flow more consistent across all three reviewed gateways.

​
2) The parity matrix exists, but not every related capability has a dedicated doc page yet

The migration guides now have a dedicated Feature Parity Matrix, but the wider docs still do not give every capability the same level of visibility. The matrix currently tracks:
  • routing transparency
  • fallback controls
  • pricing transparency
  • request transforms and defaults
  • provider/model filtering depth
The matrix closes the original documentation gap, but the wider docs still need more direct capability pages in a few places.

​
3) Some gaps are still proof gaps, not confirmed product gaps

For several backlog items, the current weakness is not necessarily “feature absent,” but “not strongly documented or validated yet”:
  • broader competitor-specific benchmark validation outside the internal benchmark tool
Treat those as follow-up validation work rather than assumed product failures.

​
What still matters most

AI Stats already has strong evidence for:
  • provider routing transparency
  • richer generation metadata
  • generation replay payload recovery through GET /v1/generations
  • more complete model/provider filters
  • better webhook and async-job ergonomics
  • more complete pricing and usage transparency
  • Vercel AI SDK compatibility across the shipped text, embedding, image, and audio surfaces
The highest-value next work is:
  1. Keep the parity matrix current as routing, pricing, async jobs, and replay/retry support evolve.
  2. Keep the competitor comparison flow current as request contracts change.
If you want to keep improving migration parity before moving fully into video review, the highest-value next step is:
  1. Keep the migration and benchmarking docs aligned as the comparison tooling evolves.
If you want to follow the broader backlog order after this review, the next major area is:
  1. Review and harden the video pipeline end to end, especially status transitions, billing state, retries, webhook behavior, and provider-access errors.
Last modified on May 19, 2026