GateKeeper · Distributed API Gateway

A distributed API gateway in Go — adds rate limiting, caching, retries, circuit breaking, and Prometheus observability to backend services across multiple replicas without changing application code.

Quick start · Architecture · Benchmarks · Failure modes

Engineering highlights

• Shared rate limits: an atomic Redis Lua script enforces one sliding-window limit across gateway replicas; each replica uses a unique member ID to avoid collisions.
• Explicit failure behavior: route-level retries and process-local circuit breakers protect upstreams; Redis failures bypass caching and rate limiting to keep requests flowing.
• Observable request paths: Prometheus metrics, a provisioned Grafana dashboard, and structured logs expose cache hits, retry attempts, dependency latency, and circuit state.
• Reproducible checks: Go race tests and a 65% internal-package coverage gate run in CI. Separate k6 workloads exercise uncached proxying, cache behavior, rate limits, upstream failure, and replicas.

Architecture

flowchart LR
  Client["Client / k6 load test"] --> LB["Nginx load balancer :8080"]
  LB --> G1["Gateway replica"]
  LB --> G2["Gateway replica"]
  LB --> G3["Gateway replica"]
  G1 --> Redis["Redis shared state"]
  G2 --> Redis
  G3 --> Redis
  G1 --> API["Mock upstream API"]
  G2 --> API
  G3 --> API
  Prom["Prometheus"] --> G1
  Prom --> G2
  Prom --> G3
  Grafana["Grafana"] --> Prom

Loading

Quickstart

Requirements: Docker with Compose v2. Go 1.23+ runs the local tests; k6 runs the load-test scripts.

git clone https://github.com/Arnavsharma2/Distributed-API-Gateway.git
cd Distributed-API-Gateway

# Install tools (macOS)
brew install go k6

# Start the full stack
docker compose up --build --scale gateway=3

Service	URL	Credentials
Gateway	http://localhost:8080	—
Prometheus	http://localhost:9090	—
Grafana	http://localhost:3000	admin / admin

# Exercise the gateway
curl -i http://localhost:8080/api/products
curl -i http://localhost:8080/api/flaky -H 'X-User-ID: demo-user'
curl -i http://localhost:8080/api/error
curl -sG http://localhost:9090/api/v1/query \
  --data-urlencode 'query=gatekeeper_requests_total'

# Unit tests
go test ./...

# Race detector, coverage, and microbenchmark
make test-race
make test-cover
make benchmark

# Load tests
k6 run loadtests/baseline.js
k6 run loadtests/cache.js
k6 run loadtests/rate_limit.js
k6 run loadtests/upstream_failure.js
k6 run loadtests/scale.js

Benchmark Results

The baseline now uses /api/benchmark/products, an intentionally uncached and unlimited route. This isolates proxy overhead instead of accidentally measuring Redis cache performance. At a fixed 2,000 RPS arrival rate, adding an Nginx upstream keep-alive pool removed per-request TCP setup and source-port exhaustion:

Single-replica baseline	Actual RPS	HTTP failures	p95 latency	p99 latency	Dropped iterations
Before upstream keep-alive	1,527.12	5.90%	215.82ms	251.41ms	13,803
After upstream keep-alive	1,992.24	0.00%	0.97ms	1.64ms	228

That is 30% more completed requests per second, no HTTP failures, and a 99.5% reduction in p95 latency under the same workload. These are fixed-arrival load tests, not claims about an absolute hardware ceiling.

The recorded local scenario suite reports:

Scenario	Replicas	Target / actual RPS	p95	p99	Scenario signal
Uncached proxy	1	2,000 / 1,992.24	0.97ms	1.64ms	0% HTTP failures
Uncached proxy	3	2,000 / 1,981.87	1.21ms	2.73ms	0% HTTP failures
Redis cache	3	1,000 / 996.70	1.33ms	2.92ms	99.85% hit rate
Global rate limit	3	200 / 200.00	1.40ms	1.99ms	Exactly 40 of 6,001 requests admitted
Sustained upstream failure	3	100 / 100.02	1.52ms	3.19ms	99.50% rejected by open circuits

The single upstream is shared by all replicas, so the three-replica result is a resilience check at this arrival rate, not a throughput multiplier. The global limiter test also verifies that Redis sorted-set members include a random replica ID; timestamp-plus-local-counter members previously collided across replicas and admitted 43 requests against a limit of 40.

Reproduce the current baseline with:

docker compose up -d --build --scale gateway=1
docker run --rm -i \
  -v "$PWD/loadtests:/scripts" \
  grafana/k6:0.54.0 run \
  -e BASE_URL=http://host.docker.internal:8080 \
  -e RATE=2000 /scripts/baseline.js

Measurements were collected on 2026-08-10 using Docker Desktop and k6 0.54.0. The test ran for 30 seconds after the stack was warm. Results will vary by host, so the scripts enforce service-level thresholds and report load-generator drops separately from HTTP failures.

Features

• Reverse proxy with YAML route configuration.
• Redis-backed sliding-window rate limiting using an atomic Lua script.
• Per-route rate-limit keys by IP, arbitrary header, or named header mode.
• GET response caching with route-level TTL.
• Retry policy with exponential backoff for transient upstream failures.
• Circuit breaker that opens after repeated failures and recovers after cooldown.
• Structured JSON logs with request ID, route, status, cache status, rate-limit status, and latency.
• Prometheus RED and dependency metrics, recording rules, alerts, and a provisioned Grafana dashboard.
• k6 scripts for baseline, rate-limit, cache, failure, and scale experiments.

Configuration

Routes are defined in deploy/docker/gateway.yaml:

routes:
  - name: products
    path_prefix: /api/products
    upstream_url: http://mock-api:3000/products
    rate_limit:
      enabled: true
      key: ip
      limit: 100
      window_seconds: 60
    cache:
      enabled: true
      ttl_seconds: 30
    retry:
      enabled: true
      attempts: 2
      base_delay_ms: 50
    circuit_breaker:
      enabled: true
      failure_threshold: 5
      cooldown_seconds: 20

Rate-limit key modes:

Mode	Behavior
key: ip	Uses X-Forwarded-For or remote IP
key: header	Uses the configured header field
key: header:X-API-Key	Reads the named header directly

If Redis is unavailable, the gateway fails open — rate limiting and cache are bypassed so availability wins over strict enforcement.

Metrics

Metric	Description
gatekeeper_requests_total	Request count by route and status
gatekeeper_requests_in_flight	Current concurrency by route and method
gatekeeper_request_duration_seconds	End-to-end latency by route, method, and status class
gatekeeper_response_size_bytes	Response-body size distribution
gatekeeper_cache_events_total	Cache hits and misses
gatekeeper_rate_limit_decisions_total	Allowed, limited, and fail-open decisions
gatekeeper_upstream_requests_total	Upstream attempts by outcome
gatekeeper_upstream_request_duration_seconds	Upstream latency by route and outcome
gatekeeper_redis_operations_total	Redis operations and errors by operation
gatekeeper_redis_operation_duration_seconds	Redis dependency latency
gatekeeper_retries_total	Retry attempts by route and reason
gatekeeper_circuit_state	One-hot closed, open, and half-open state
gatekeeper_circuit_transitions_total	Circuit state changes
gatekeeper_circuit_rejected_total	Requests rejected by open circuits

The request histogram starts at 0.5ms, so Prometheus can resolve the low-millisecond percentiles produced by the local benchmark. The Grafana dashboard covers request rate, 5xx ratio, p50/p95/p99 latency, concurrency, cache effectiveness, upstream latency, Redis latency, rate-limit decisions, retries, and circuit behavior. Its route variable supports both focused debugging and fleet-wide views.

Prometheus also provisions five alerts: target down, high 5xx ratio, high p95 latency, Redis errors, and an open circuit. Three recording rules precompute the 5-minute request rate, 5xx ratio, and p95 latency used for SLO monitoring.

Failure Modes

Failure	Behavior
Redis unavailable	Fails open — rate limiting and cache bypass; requests continue to upstream
Upstream intermittent 5xx	Retries with exponential backoff; records retry and error metrics per attempt
Upstream sustained failure	Circuit breaker opens after threshold; returns 503 until cooldown half-open trial
Multiple gateway replicas	All replicas share Redis state — limits are enforced globally, not per process