This is a running LLM gateway: it routes, fails over, rate-limits and meters every request, behind an OpenAI-compatible API. The default request uses simulated providers. Real-model controls appear only when this deployment has a matching configured route; configuration does not prove upstream availability or pricing.
Route availability is shown below after metadata loads.
connecting ...
Every served request crosses the five stages above; a click sweeps them, reports the measured total latency, and leaves each stage label carrying its measured share of that request (the gateway's Server-Timing header). The sweep itself is illustrative, and a refused request lights nothing: the page says only what the response says.
The first button (simulated) sends the prompt above - or a fixed one-liner when it is empty - through the running gateway (auth, admission, routing, failover, metering) to the mock providers. Custom prompts stay on the simulated route; any real-model controls use a fixed one-line prompt. It uses an in-process visitor identity, not a reusable bearer key: max 24 tokens, at most 10 per minute per address, with its own small shared budget, so clicking fast just gets 429s. No spend, nothing outside this demo.
This demo is also an API you can drive. Mint a temporary key and call the gateway from any OpenAI-compatible client: your request crosses the same auth, admission, routing, failover and metering, and lands in the live log below. Keys reach the simulated fleet only and expire after 24 hours. They never grant real-model access; rejection of an unavailable alias does not prove a real route is configured.
Each key: 0.5 requests/s and 20 tokens/s sustained with a small burst, simulated models only, 24-hour expiry. Minting is bounded too: a few keys per hour per address and a fixed daily total across all visitors, enforced in the shared store. The key is shown once and stored only as a SHA-256 digest server-side, so it cannot be recovered - mint a fresh one if you lose it. When the shared store is not configured, keys live only in the process that minted them and the mint response says so.
Live counters from the gateway's own metering. Counters load once when the page opens; after that nothing polls on its own. Refresh now pulls once; Watch live polls every 5s for up to 60 seconds and only while this tab is visible, then stops - a forgotten tab cannot keep spending the store's daily allowance. The gateway caches each aggregate for a few seconds so viewers on the same instance share one store read, and the edge rule limits each IP to 60 requests/minute; an absolute cap on store spend is the store plan's own hard quota, not something this page can enforce. A pulse on an edge means that provider's completed-request counter went up since the last refresh; these aggregate counters cannot show the exact path of any single request (a request that failed over may count on both providers).
One click runs a complete OAuth 2.0 Token Exchange (RFC 8693) flow inside the gateway and proves its consequences through the real request path: a user token is exchanged down to a parent agent, then narrowed further to a sub-agent that may reach the plain simulated model (gpt-sim) but is REFUSED the expensive one (gpt-sim-pro) with 403 insufficient_scope - the same aliases the Try buttons above use, all simulated, no real model, no spend. A widening attempt by the sub-agent is denied at the exchange, and every grant and denial lands in a hash-chained audit log whose integrity is verified before the result renders. Honest limits: the issuer is embedded in this process (one trust domain - no remote issuer or JWKS); the demo tokens are explicitly unbound bearer tokens and proof-bound (cnf) tokens are rejected rather than downgraded; each run issues and consumes its own tokens, so serverless cold starts change nothing. This demonstrates delegation semantics and least-privilege enforcement; it is not production authorization.
| time | key | served by | status | latency | tokens (p+c) | est. cost | attempts |
|---|---|---|---|---|---|---|---|
| no requests recorded yet | |||||||
The newest 20 requests the gateway served, newest first, from its own request log. Prompt and reply content is never logged - token counts only - and keys appear as their public labels, never secrets. A simulated row was answered by a mock provider; a real row by a live upstream. Same refresh cadence as the counters above; the endpoint is GET /demo/events.
GET /healthz | liveness + per-provider breaker state; no key needed |
GET /demo/stats | the aggregated counters this page polls; no key, no per-key data |
GET /demo/events | the recent-request feed above; no key, no content, no secrets |
GET /v1/models | lists the demo model gpt-sim; needs a key |
POST /v1/chat/completions | OpenAI-shaped chat completion; add "stream": true for SSE. Responses carry a measured per-stage Server-Timing header (auth, admission, routing, provider; metering too on non-stream) |
GET /metrics, GET /admin/usage | Prometheus metrics and per-key usage; admin key only |
curl https://llmgate-demo.vercel.app/healthz
curl -H "Authorization: Bearer YOUR_DEMO_KEY" -H "content-type: application/json" -d '{"model":"gpt-sim","messages":[{"role":"user","content":"hello"}]}' https://llmgate-demo.vercel.app/v1/chat/completions
Run a few completions and watch them pulse through the diagram above. The demo key is shared privately, not published here. Edge protection: 60 requests/minute per IP, then HTTP 429. The metering status above reports whether this deployment uses a shared store or per-instance memory. Shared-store limits coordinate function instances; in-memory limits do not. Circuit-breaker state stays per instance. The cost figure prices simulated usage at the demo's simulated list prices - an estimate, not billing. This is a demo, not a production-ready service. Provider and hosting charges depend on the account and are not verified by this page.