Skip to content

Latest commit

 

History

15 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Sentinel

Sentinel is a multi-provider AI Gateway designed to sit between applications and LLM providers, providing a single API interface while handling reliability, performance, cost, and traffic-management concerns.

Instead of an application communicating directly with a single LLM provider, requests are sent to Sentinel through a unified /chat endpoint. Sentinel decides how the request should be handled and applies a series of infrastructure layers before communicating with the selected provider.

The current implementation supports Groq as the primary provider and OpenAI as a secondary provider/fallback.

Current Architecture

Client
  │
  ▼
Sentinel /chat
  │
  ├── Request Queue / Backpressure
  │
  ├── Exact + Semantic Cache
  │
  ├── Token Bucket Rate Limiter
  │
  ├── Circuit Breaker
  │
  ├── Provider Routing
  │
  ├───────────────┐
  ▼               ▼
Groq           OpenAI
Primary        Fallback
  │               │
  └───────┬───────┘
          ▼
       Response

Implemented Features

Unified AI Gateway

Applications interact with a single POST /chat endpoint rather than integrating separately with each LLM provider.

Requests can specify a provider, while Groq is used as the default provider.

Multi-Provider Routing

Sentinel currently integrates:

  • Groq : primary provider
  • OpenAI (gpt-4o-mini) : secondary provider and automatic fallback

The provider layer is designed so additional providers can be introduced later without changing the gateway's public API.

Token Bucket Rate Limiting

Sentinel uses an in-memory token bucket to control request bursts.

The current configuration allows an initial burst of 5 requests and refills at 1 token per second.

Requests that exceed the available tokens are rejected with HTTP 429 Too Many Requests.

Exact-Match Caching

Responses are cached using a SHA-256 hash derived from the provider and prompt.

Identical requests can therefore be served directly from memory without contacting the LLM provider again.

Semantic Caching

Sentinel extends exact-match caching with embedding-based semantic similarity.

Prompts are converted into embeddings using all-MiniLM-L6-v2, and cosine similarity is used to identify semantically similar requests.

A similarity threshold of 0.80 was selected based on testing.

This allows requests with different wording but similar meaning to reuse previously generated responses.

Circuit Breaker

Sentinel implements a three-state circuit breaker for the primary Groq provider:

CLOSED → OPEN → HALF_OPEN → CLOSED

Repeated Groq failures open the circuit and prevent further requests from being sent to the unhealthy provider until the recovery timeout expires.

Automatic Provider Failover

When Groq fails, Sentinel automatically falls back to OpenAI rather than returning the provider failure directly to the client.

This provides provider-level resilience and allows Sentinel to continue serving requests during an upstream outage.

Request Queue and Backpressure

Sentinel includes an in-memory request queue that limits the number of requests being processed concurrently.

When the configured capacity is reached, additional requests are rejected with HTTP 503 Service Unavailable rather than allowing uncontrolled request growth.

Request cleanup is guaranteed using try/finally so queue capacity is released even when request processing fails.

Load-Testing Harness

A Python load-testing client using asyncio and httpx generates concurrent requests against Sentinel and records:

  • HTTP status code
  • Cache status
  • Failover status
  • Individual request latency
  • Aggregate statistics

This provides a reproducible way to measure Sentinel's behavior under burst traffic and provider failures.

Benchmarks

Benchmarks below are observed results from local load tests using 10 concurrent requests. They are experimental measurements rather than universal performance guarantees.

Cache Performance

Using the same 10-request workload:

Metric Result
Cache hit rate on repeated workload 100.00%
Average latency with cache hits 424.89 ms
Average successful-request latency with cache disabled 4407.02 ms
Observed latency reduction ~90.4%

The cache-enabled workload served repeated requests without contacting the LLM provider.

The cache-disabled comparison required provider calls for the successful requests, demonstrating the latency benefit of reusing cached responses.

Semantic-cache testing also verified that sufficiently similar prompts could reuse cached responses at a configured similarity threshold of 0.80.

Rate-Limiter Burst Test

A burst of 10 concurrent requests was tested with the token bucket enabled and disabled.

Configuration Successful Rejected
Limiter disabled 10 0
Limiter enabled 5 5

With the limiter enabled, 50% of the burst was throttled with HTTP 429 Too Many Requests.

This confirmed that the token bucket correctly limits burst traffic according to its configured capacity.

Provider Failover Test

Groq was deliberately made unavailable during testing to simulate an upstream provider failure.

Metric Observed Result
Failover attempts 5
Successful fallback requests 5
Failover success rate 100.00%
Average Groq failure detection time 367.15 ms
Average failover transition time 0.30 ms
Average OpenAI fallback duration 3253.45 ms
Average total recovery time 3253.76 ms

The results show that Sentinel detected Groq failures quickly and transitioned to OpenAI almost immediately. Most of the recovery time was spent waiting for the fallback provider to generate its response.

The 100% failover success rate refers to requests that were admitted by the rate limiter and actually entered the Groq → OpenAI failover path.

Queue / Backpressure Test

The request queue was tested with a deliberately small capacity and simulated long-running requests.

The test verified that:

  • Requests within queue capacity were accepted.
  • Requests arriving after the queue reached capacity were rejected with HTTP 503 Service Unavailable.
  • Queue capacity was released correctly after requests completed.
  • try/finally cleanup prevented requests from remaining permanently stuck in the queue after failures.

Current Status

Sentinel currently provides:

  • Unified LLM API
  • Groq + OpenAI provider routing
  • Token bucket rate limiting
  • Exact-match caching
  • Semantic caching
  • Circuit breaker
  • Automatic provider failover
  • In-memory request queue and backpressure
  • Async concurrent load testing
  • Measured performance and reliability benchmarks

How to Run

1. Clone the repository

git clone https://github.com/Ritesh90256/Sentinel.git
cd Sentinel

2. Create and activate the virtual environment

Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1

3. Install dependencies

pip install -r requirements.txt

4. Configure environment variables

Create a .env file in the project root:

GROQ_API_KEY=your_groq_api_key
OPENAI_API_KEY=your_openai_api_key
RATE_LIMIT_ENABLED=true

5. Start Sentinel

uvicorn app.main:app --reload

The API will be available at:

http://127.0.0.1:8000

Interactive API documentation is available at:

http://127.0.0.1:8000/docs

6. Run the load-testing harness

With Sentinel running in one terminal, open another terminal, activate the virtual environment, and run:

python tests/load_test.py

The load-testing script sends concurrent requests to Sentinel and reports response status, cache behavior, failover behavior, and latency.

Final Summary

Sentinel is a personal project focused on building an infrastructure-oriented AI Gateway that sits between applications and multiple LLM providers. Instead of requiring applications to handle provider-specific APIs, failures, traffic limits, and caching independently, Sentinel provides a unified /chat endpoint and manages these concerns centrally.

The project currently supports Groq as the primary provider and OpenAI as the fallback provider.

Sentinel combines several backend infrastructure patterns:

  • Unified provider routing through a single API endpoint
  • Token-bucket rate limiting to control burst traffic
  • Exact-match caching using hashed request keys
  • Semantic caching using embeddings and cosine similarity
  • Circuit breaking to stop repeatedly calling an unhealthy provider
  • Automatic Groq → OpenAI failover when the primary provider fails
  • In-memory request queuing and backpressure to prevent uncontrolled request growth
  • Asynchronous load testing using Python, asyncio, and httpx

The project was also tested rather than only implemented. Controlled experiments measured cache effectiveness, burst throttling, and provider failure recovery. The observed results demonstrated a 100% cache hit rate on the repeated workload, approximately 90.4% lower measured latency for cached responses, 5 of 10 burst requests throttled by the configured token bucket, and a 100% successful fallback rate among requests that entered the Groq → OpenAI failover path.

The load-testing and failure experiments also helped identify where time was actually being spent. In the provider-failure test, Sentinel detected Groq failures in an average of 367.15 ms and transitioned to OpenAI in approximately 0.30 ms, while most of the recovery time came from the fallback provider's response itself.

Overall, Sentinel evolved from a simple LLM API wrapper into a small but complete AI infrastructure gateway, combining performance optimization, traffic management, and provider resilience behind a single interface.

About

An intelligent AI Gateway for routing, caching, rate limiting, and failover across multiple LLM providers.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages