AI request gateway and usage dashboard
Internal AI infrastructure
The AI request gateway is internal infrastructure Shroti Host built to route requests from its own software across AI model providers. It tracks rate limits and usage windows, fails over automatically when a route is unavailable, and shows usage on a dashboard. It is Postgres-backed, deployed on Vercel with host workers, and not client work.
This is a private system, so there is no public link.
Screenshots

A screen from a demo copy of the dashboard with made-up keys and usage. The live dashboard is private.
Diagram
- 1
Shroti Host software
Sends an AI request to the gateway
- 2
Router
Checks rate limits and usage windows for each route
- 3
Model provider
The request goes to an available route
- 4
Failover
If a route is unavailable, the request moves to the next one
- 5
Usage dashboard
Usage is recorded in Postgres and shown on the dashboard
What problem does the AI request gateway solve?
Software that calls AI models stops working when a provider hits a rate limit or has an outage, and usage is hard to see across providers.
What does the AI request gateway do?
- Request routing across AI model providers
- Rate-limit and usage-window awareness
- Automatic failover when a route is unavailable
- A usage dashboard
- API keys for Shroti Host's own tools, each with a scope and a daily limit
How did Shroti Host build the AI request gateway?
Shroti Host built the gateway on Vercel with Postgres for state and workers on its own hosts.
- Routing decisions take current rate limits and usage windows into account before a request is sent
- Failover is automatic, so the software behind it keeps working when one route is unavailable
What does the AI request gateway show about working with Shroti Host?
Shroti Host builds the infrastructure that AI features depend on: routing, limits, failover and visibility, not just the feature on top.
Services this project is evidence for