# AI request gateway and usage dashboard

Built by Shroti Host. Status: In use internally. Internal AI infrastructure.

> The AI request gateway is internal infrastructure Shroti Host built to route requests from its own software across AI model providers. It tracks rate limits and usage windows, fails over automatically when a route is unavailable, and shows usage on a dashboard. It is Postgres-backed, deployed on Vercel with host workers, and not client work.

URL: https://shrotihost.in/portfolio/ai-request-gateway

- Type: Internal infrastructure
- Built by: Shroti Host
- Status: In use internally (private)
- Runs on: Vercel with host workers, backed by Postgres
- Stack: Postgres, Vercel, Host workers

## What problem does the AI request gateway solve?

Software that calls AI models stops working when a provider hits a rate limit or has an outage, and usage is hard to see across providers.

## What does the AI request gateway do?

- Request routing across AI model providers
- Rate-limit and usage-window awareness
- Automatic failover when a route is unavailable
- A usage dashboard
- API keys for Shroti Host's own tools, each with a scope and a daily limit

## How did Shroti Host build the AI request gateway?

Shroti Host built the gateway on Vercel with Postgres for state and workers on its own hosts.

- Routing decisions take current rate limits and usage windows into account before a request is sent
- Failover is automatic, so the software behind it keeps working when one route is unavailable

## How a request moves through the gateway

1. Shroti Host software: Sends an AI request to the gateway
2. Router: Checks rate limits and usage windows for each route
3. Model provider: The request goes to an available route
4. Failover: If a route is unavailable, the request moves to the next one
5. Usage dashboard: Usage is recorded in Postgres and shown on the dashboard

## Screenshots

- API keys: each of Shroti Host's own tools gets a key with its own scope and daily limit. Demo data. https://shrotihost.in/work/ai-gateway-keys.webp

A screen from a demo copy of the dashboard with made-up keys and usage. The live dashboard is private.

## What does the AI request gateway show about working with Shroti Host?

Shroti Host builds the infrastructure that AI features depend on: routing, limits, failover and visibility, not just the feature on top.

Services: SaaS development (https://shrotihost.in/saas-development).
Built by Shroti Host, founded by Priyanshu Shroti (https://shrotihost.in/about/priyanshu-shroti).
