Skip to content
Shroti Host
Built by Shroti HostIn use internally

AI request gateway and usage dashboard

Internal AI infrastructure

The AI request gateway is internal infrastructure Shroti Host built to route requests from its own software across AI model providers. It tracks rate limits and usage windows, fails over automatically when a route is unavailable, and shows usage on a dashboard. It is Postgres-backed, deployed on Vercel with host workers, and not client work.

This is a private system, so there is no public link.

Screenshots

AI request gateway API keys page with demo data: a form to create a key, and four demo keys for tools such as nightly reports and a support desk, with their scope, route setting, daily limit and 30-day request count.
API keys: each of Shroti Host's own tools gets a key with its own scope and daily limit. Demo data.

A screen from a demo copy of the dashboard with made-up keys and usage. The live dashboard is private.

Diagram

How a request moves through the gateway
  1. 1

    Shroti Host software

    Sends an AI request to the gateway

  2. 2

    Router

    Checks rate limits and usage windows for each route

  3. 3

    Model provider

    The request goes to an available route

  4. 4

    Failover

    If a route is unavailable, the request moves to the next one

  5. 5

    Usage dashboard

    Usage is recorded in Postgres and shown on the dashboard

What problem does the AI request gateway solve?

Software that calls AI models stops working when a provider hits a rate limit or has an outage, and usage is hard to see across providers.

What does the AI request gateway do?

  • Request routing across AI model providers
  • Rate-limit and usage-window awareness
  • Automatic failover when a route is unavailable
  • A usage dashboard
  • API keys for Shroti Host's own tools, each with a scope and a daily limit

How did Shroti Host build the AI request gateway?

Shroti Host built the gateway on Vercel with Postgres for state and workers on its own hosts.

  • Routing decisions take current rate limits and usage windows into account before a request is sent
  • Failover is automatic, so the software behind it keeps working when one route is unavailable

What does the AI request gateway show about working with Shroti Host?

Shroti Host builds the infrastructure that AI features depend on: routing, limits, failover and visibility, not just the feature on top.

Services this project is evidence for