Chapter 20: API Gateways & Reverse Proxies

Reverse vs Forward Proxy

API Gateway Pattern

Why Centralize? Cross-cutting concerns (auth, rate limiting, logging) duplicated across services create maintenance burden, inconsistent behavior, and security gaps. Gateway provides single enforcement point.

BFF Pattern: Separate gateway per client type (mobile, web, IoT). Each gateway aggregates, transforms data optimally for its client. Mobile gateway returns compact JSON, handles high latency. Web gateway returns richer data.

Core Functions

Rate Limiting

Token Bucket Algorithm

Distributed Rate Limiting

• Redis Centralized: All gateways check Redis for token count. Atomic INCR/DECR ensures consistency. Single point of truth but adds latency.
• Local + Sync: Each gateway maintains local counter, periodically syncs with Redis. Lower latency, eventual consistency, potential overage.
• Sticky Sessions: Route same API key to same gateway instance. No coordination needed but uneven load distribution.

Auth Offloading

JWT Validation at Gateway: Gateway verifies JWT signature, expiry, issuer. Extracts claims (user ID, roles), forwards as headers to backend. Backend trusts gateway, skips re-validation.

API Key Management

• Storage: Store hashed keys in database/cache (Redis). Never log plaintext keys.
• Rotation: Support dual keys during rotation window. Deprecate old key after grace period.
• Scopes: Attach permissions to keys (read-only, admin). Gateway enforces before routing.

Routing Strategies

Canary Deployments

Route small percentage of traffic to new version. Monitor error rates, latency. Gradually increase percentage or rollback if issues detected.

# Kong canary config
services:
  - name: order-service-v1
    weight: 95
  - name: order-service-v2
    weight: 5

Implementations

Gateway vs Service Mesh

Use Both: Gateway handles external traffic (auth, rate limit). Service mesh handles internal traffic (mTLS, retries, circuit breaking). Gateway can integrate with mesh for unified observability.

API Versioning

Best Practice: URL path versioning most common. Reserve major versions for breaking changes. Use feature flags for minor changes within version.

Edge Security

Web Application Firewall (WAF)

• SQL Injection: Detect patterns like ' OR 1=1-- in query params, body
• XSS: Block <script> tags, JavaScript event handlers in input
• Path Traversal: Reject ../../../etc/passwd attempts
• OWASP Top 10: Rulesets for common web vulnerabilities

DDoS Protection

• Rate Limiting: Aggressive limits for unauthenticated traffic
• SYN Flood: SYN cookies, connection limits at L4
• HTTP Flood: Challenge-response (CAPTCHA), JS challenge
• CDN: Distribute load globally (Cloudflare, Akamai absorb volumetric attacks)

Bot Detection

• Behavioral Analysis: Mouse movement, keystroke timing
• Fingerprinting: Browser, device, IP reputation
• CAPTCHA: reCAPTCHA v3 (invisible score-based)
• Rate Anomalies: Flag IPs making 1000s of requests

Mutual TLS (mTLS)

• Client Certificates: Gateway validates client cert before allowing connection
• Zero Trust: Every service authenticates, not just perimeter
• Service Mesh: Automatic mTLS between services (Istio, Linkerd)

IP Whitelisting/Blacklisting

• Whitelist: Only allow traffic from trusted IPs (partner APIs, internal tools)
• Blacklist: Block known malicious IPs (threat intel feeds)
• Geo-Blocking: Block entire countries if not serving those regions

Summary

Rate Limiting Implementation

Sliding Window Counter Algorithm

FUNCTION is_allowed(user_id, limit, window_sec):
    current_window = FLOOR(current_time / window_sec)
    key = "rate:" + user_id + ":" + current_window

    // Increment counter for current window
    count = REDIS.INCR(key)
    IF count == 1:
        REDIS.EXPIRE(key, window_sec * 2)  // Keep for 2 windows

    // Sliding window: weighted average of previous + current
    previous_window = current_window - 1
    prev_key = "rate:" + user_id + ":" + previous_window
    prev_count = REDIS.GET(prev_key) OR 0

    // Calculate elapsed time in current window
    elapsed = current_time MOD window_sec
    weight = (window_sec - elapsed) / window_sec

    // Effective count = weighted previous + current
    effective_count = (prev_count * weight) + count

    RETURN effective_count <= limit

Example Walkthrough

Redis Implementation

# Atomic sliding window check in Redis (Lua script)
local current_key = KEYS[1]
local previous_key = KEYS[2]
local limit = tonumber(ARGV[1])
local window_sec = tonumber(ARGV[2])
local weight = tonumber(ARGV[3])

local current = redis.call('INCR', current_key)
if current == 1 then
    redis.call('EXPIRE', current_key, window_sec * 2)
end

local previous = tonumber(redis.call('GET', previous_key) or 0)
local effective = (previous * weight) + current

if effective <= limit then
    return {1, effective}  -- allowed
else
    redis.call('DECR', current_key)  -- rollback increment
    return {0, effective}  -- denied
end

Performance: Redis INCR is O(1), entire check completes in 1-2ms at P99. For 10K requests/sec, needs ~20MB memory per million users (2 keys * 8 bytes * 1M users).

Circuit Breaker at Gateway

Purpose: Prevent cascading failures when backend service degrades. Gateway fails fast instead of waiting for timeouts, reducing resource exhaustion.

State Machine Logic

CLASS CircuitBreaker:
    state = CLOSED
    failure_count = 0
    success_count = 0
    last_failure_time = null
    request_log = []  // Last N requests

    FUNCTION call(request):
        IF state == OPEN:
            IF current_time - last_failure_time > wait_duration:
                state = HALF_OPEN
                success_count = 0
            ELSE:
                RETURN error("503 Service Unavailable - Circuit Open")

        IF state == HALF_OPEN:
            IF success_count >= permitted_calls_in_half_open:
                state = CLOSED
                failure_count = 0
                request_log = []

        TRY:
            response = backend.call(request, timeout=slow_call_threshold)
            record_success()
            RETURN response
        CATCH TimeoutException, ServerError:
            record_failure()
            IF should_open_circuit():
                state = OPEN
                last_failure_time = current_time
            THROW error

    FUNCTION record_failure():
        request_log.append({success: false, time: current_time})
        request_log = request_log[-10:]  // Keep last 10

    FUNCTION should_open_circuit():
        IF request_log.length < 10: RETURN false
        failures = COUNT(request_log WHERE success == false)
        RETURN failures / request_log.length >= failure_threshold

Configuration Parameters

Real-World Example

Scenario: Payment service starts timing out (database overload). Gateway circuit breaker opens after 5/10 requests timeout. For next 30 seconds, gateway returns 503 immediately without calling payment service. After 30s, allows 3 probe requests. If successful, closes circuit and resumes normal traffic.

Impact: Without circuit breaker, all gateway threads block on payment timeouts (5s each), exhausting thread pool. With circuit breaker, gateway remains responsive, other services unaffected.

Request/Response Transformation

Header Manipulation

Request Headers (Gateway → Backend)

// Strip internal/sensitive headers before forwarding
REMOVE: X-Forwarded-Proto, X-Real-IP, Authorization
ADD:    X-Request-ID: generate_uuid()
        X-User-Id: extract_from_jwt(Authorization)
        X-Client-IP: client.ip_address
        X-Gateway-Time: current_timestamp()

// Example transformation
BEFORE (client → gateway):
    GET /api/orders
    Authorization: Bearer eyJhbGc...
    User-Agent: Mozilla/5.0
    X-Custom-Header: untrusted_value

AFTER (gateway → backend):
    GET /api/orders
    X-Request-ID: 550e8400-e29b-41d4-a716-446655440000
    X-User-Id: 12345
    X-Roles: admin,user
    X-Client-IP: 203.0.113.42
    User-Agent: Mozilla/5.0
    // Authorization removed, X-Custom-Header removed

Response Headers (Backend → Gateway → Client)

// Remove internal details
REMOVE: X-Backend-Server, X-Database-Query-Time, X-Internal-Cache-Hit

// Add security headers
ADD:    X-Content-Type-Options: nosniff
        X-Frame-Options: DENY
        X-XSS-Protection: 1; mode=block
        Strict-Transport-Security: max-age=31536000; includeSubDomains

// Add CORS headers (if configured)
ADD:    Access-Control-Allow-Origin: https://trusted-domain.com
        Access-Control-Allow-Methods: GET, POST, PUT, DELETE
        Access-Control-Allow-Headers: Content-Type, Authorization
        Access-Control-Max-Age: 3600

Response Aggregation (BFF Pattern)

Problem: Mobile app needs user profile, order history, and recommendations. Without gateway, makes 3 sequential API calls (high latency on cellular).

// Gateway aggregation endpoint
GET /api/mobile/dashboard

GATEWAY LOGIC:
    // Parallel backend calls
    PARALLEL:
        user = GET https://user-service/api/users/{user_id}
        orders = GET https://order-service/api/orders?user_id={user_id}&limit=5
        recommendations = GET https://recommendation-service/api/recommendations/{user_id}

    // Combine responses
    RETURN {
        "user": {
            "name": user.name,
            "avatar": user.avatar_url,
            "tier": user.membership_tier
        },
        "recent_orders": orders.items.map(o => {
            "id": o.id,
            "status": o.status,
            "total": o.total_amount
        }),
        "recommended_products": recommendations.slice(0, 3)
    }

// Result: 3 backend calls in parallel (200ms), 1 client round-trip
// vs 3 sequential client calls (600ms + mobile latency)

Protocol Conversion

Field Mapping & Data Enrichment

// Backend returns internal field names
BACKEND RESPONSE:
{
    "usr_id": 12345,
    "crt_dt": "2026-05-12T10:30:00Z",
    "ord_amt": 5999,
    "curr_cd": "USD"
}

// Gateway transforms to client-friendly names + enrichment
GATEWAY RESPONSE:
{
    "user_id": 12345,
    "created_date": "2026-05-12T10:30:00Z",
    "order_amount": 59.99,  // cents to dollars
    "currency": "USD",
    "formatted_amount": "$59.99",  // added by gateway
    "created_relative": "2 hours ago"  // computed
}

Production Challenges & Solutions

Capacity Planning

Gateway Observability

Why Critical: Gateway sees ALL traffic, making it ideal aggregation point for metrics. Centralized observability eliminates per-service instrumentation duplication.

Key Metrics to Monitor

Distributed Tracing

// Gateway adds trace context to all backend requests
REQUEST from client:
    GET /api/orders

GATEWAY forwards with trace headers:
    GET /api/orders
    X-Request-ID: 550e8400-e29b-41d4-a716-446655440000
    X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7
    X-B3-SpanId: e457b5a2e4d86bd1
    X-B3-Sampled: 1

// Trace spans:
1. Gateway receives request: 0ms
2. Gateway auth check: 0-3ms
3. Gateway → Order Service: 3-150ms
   - Order Service DB query: 10-140ms
4. Gateway → Inventory Service: 3-80ms (parallel)
5. Gateway response aggregation: 150-155ms
6. Gateway sends response: 155ms

// Trace shows: Total 155ms, Order Service DB query is bottleneck (130ms)

Logging Best Practices

Dashboard Example Panels

• Traffic Overview: Total req/sec, 2xx/4xx/5xx breakdown (stacked area chart)
• Latency Heatmap: Histogram over time, color-coded by percentile
• Top Slow Routes: Table sorted by P99 latency, link to trace examples
• Error Rate by Service: Line chart per backend service, alert threshold line
• Circuit Breaker Status: Current state per service (CLOSED/OPEN/HALF_OPEN), time in state
• Rate Limit Hit Rate: Percentage of requests rate-limited, by user tier (free vs paid)
• Active Connections: Gauge showing current vs max pool size, per backend

Alerting Strategy