Skip to main content

API Gateway & Routing

Intelligent reverse proxy with dynamic routing, permission verification, rate limiting, circuit breaking, and request transformation.

Scope

This document covers the following packages and their interfaces:

LayerPackagesKey Files
Applicationpkg/v2/application/proxy/proxy_service.go, service.go, transformer.go, error_handler.go, client.go
Domain Modelspkg/v2/domain/route/registry.go, registry_inmem.go, route.go, spec.go
Domain Servicespkg/v2/domain/service/route_conflict_detector.go
Infrastructurepkg/v2/infrastructure/circuitbreaker/, pkg/v2/infrastructure/ratelimit/manager.go, limiter.go, middleware.go
Middlewarepkg/v2/infrastructure/middleware/permission.go, cors.go
Presentationpkg/v2/presentation/http/proxy_handler.go
Serverpkg/v2/server/http.go (middleware pipeline)

Overview

Based on pkg/v2/application/proxy/, the API Gateway provides:

  • Intelligent Routing: Dynamic path-to-upstream URL mapping with pattern matching
  • Permission Verification: OpenFGA authorization checks before forwarding requests
  • Rate Limiting: Hierarchical limits (route-level + per-user)
  • Circuit Breaking: Per-upstream failure protection to prevent cascading failures
  • Request Transformation: Security-first header forwarding with opt-out model
  • Streaming Support: Efficient handling of large payloads and file uploads
  • Error Handling: Comprehensive error classification and status mapping
  • Health Checking: Monitor backend availability
  • Telemetry: Request tracing, metrics, and logging

Architecture

Layered Structure

Client Request

HTTP Server (port 8080)

Middleware Pipeline (logging, auth, CORS)

Permission Middleware (OpenFGA check)

Rate Limit Middleware (route + user limits)

Proxy Handler

Proxy Service (routing, transformation)

Circuit Breaker (per-upstream protection)

HTTP Client (connection pooling)

Upstream Service

Core Components

ProxyService (pkg/v2/application/proxy/proxy_service.go:15-23)

type proxyService struct {
registry route.Registry // Route matching and lookup
client *http.Client // HTTP client with connection pooling
transformer *Transformer // Request/response transformation
errorHandler *ErrorHandler // Error classification and mapping
breakerManager circuitbreaker.Manager // Per-upstream circuit breakers
logger telemetry.Logger // Structured logging
}

Dependencies & Interactions:

  • → Route Registry (in-memory): Fast route matching with pattern support
  • → HTTP Client: Connection pooling, HTTP/2, configurable timeouts
  • → Transformer: Header security rules, proxy header injection
  • → Error Handler: Status code mapping, error envelope wrapping
  • → Circuit Breaker Manager: Per-upstream failure tracking and fast-fail
  • → Permission Checker (middleware): OpenFGA authorization via multi-level cache
  • → Rate Limiter (middleware): Redis-based sliding-window counter algorithm
  • → Telemetry: Distributed tracing, metrics collection, structured logging

Service Interface

From pkg/v2/application/proxy/service.go:9-15:

type Service interface {
// ProxyRequest - buffered proxy with structured request/response
ProxyRequest(ctx context.Context, req *ProxyRequest) (*ProxyResponse, error)

// ProxyHTTP - streaming proxy with native HTTP response writer
ProxyHTTP(ctx context.Context, w http.ResponseWriter, r *http.Request) error

// GetRoute - route lookup utility
GetRoute(ctx context.Context, path string, method route.HTTPMethod) (*route.RegisteredRoute, error)
}

Two Proxy Modes:

  • Buffered (ProxyRequest): For API integration, returns structured response
  • Streaming (ProxyHTTP): For HTTP handlers, writes directly to response writer, efficient for large payloads

Request Flow Pipeline

Complete request lifecycle from ProxyHTTP (proxy_service.go:170-300):

StepFunctionFile:LinesDescription
1Route matchingproxy_service.go:180-186registry.Match(path, method) finds RegisteredRoute + RouteSpec
2Upstream URL constructionproxy_service.go:190-195GetUpstreamURL() handles StripBasePath logic
3Request transformationproxy_service.go:197-203Header security rules, proxy headers injection
4Timeout configurationproxy_service.go:205-210Hierarchical: RouteSpec → Route → Default (30s)
5Circuit breaker executionproxy_service.go:99-115Per-upstream breaker with hybrid 5xx handling
6Response handlingproxy_service.go:240-280Conditional: 5xx wrapped, <5xx streamed
7Metrics & tracingproxy_service.go:140-165Counters, histograms, span attributes

Route Registry & Matching

Based on pkg/v2/domain/route/registry_inmem.go:

In-Memory Registry

inMemoryRegistry (lines 11-17):

type inMemoryRegistry struct {
mu sync.RWMutex // Thread-safe concurrent access
routes map[uuid.UUID]*RegisteredRoute // Route ID → Route
}

Thread Safety:

  • RLock() for concurrent reads (hot path - every request)
  • Lock() for exclusive writes (registration/deregistration)
  • Used in every incoming request routing decision

Route Matching

From registry_inmem.go:Match() (lines 45-120):

Matching Algorithm:

1. Iterate all registered routes
2. For each route, check all RouteSpecs
3. Match pattern type (Exact, Prefix, Regex, Parameterized)
4. Match HTTP method
5. Return first match (routes sorted by specificity)

RegisteredRoute Structure

From pkg/v2/domain/route/route.go:

type RegisteredRoute struct {
ID uuid.UUID // Unique route ID
AppID uuid.UUID // App owning route
AppName string // App name (e.g., "todos-app")
BasePath string // e.g., "/api/apps/todos"
UpstreamBaseURL string // http://todos-bff.svc:8080
StripBasePath bool // Remove BasePath before forwarding
IsPublic bool // Skip auth if true
Timeout *config.TimeoutConfig
Headers *config.HeaderConfig
RouteSpecs []*Spec // Fine-grained routing rules
Permissions []string // Required OpenFGA scopes
RateLimit *config.RateLimit // API-level limits
HealthCheck *config.HealthCheckConfig
}

RouteSpec (Fine-grained routing rules)

type Spec struct {
Pattern string // "/api/apps/todos/**"
PatternType PatternType // Exact|Prefix|Regex|Parameterized
Methods []HTTPMethod // [GET, POST, PUT, DELETE]
Scopes []string // For permission checking
Timeout *config.TimeoutConfig // Overrides route-level
RateLimit *config.RateLimit // Overrides route-level
IsPublic bool // Overrides route-level
}

Pattern Types

PatternTypeExact: /api/items

  • Matches exact path only
  • Example: /api/items matches, /api/items/1 does not

PatternTypePrefix: /api/items/** or /api/items/*

  • **: Matches any path with prefix including trailing slashes
  • *: Matches single path segment

PatternTypeParameterized: /api/:id/items

  • :id extracts parameter from path
  • Useful for RESTful routes

PatternTypeWildcard: /api/*/items

  • * matches any single segment
  • Example: /api/v1/items, /api/v2/items

PatternTypeRegex: regex:^/api/.*

  • Full regex pattern matching
  • Most flexible but slower

Upstream URL Construction

With StripBasePath=true:

Input Path: /api/apps/todos/items
BasePath: /api/apps/todos
UpstreamBaseURL: http://todos-bff.svc:8080

Result: http://todos-bff.svc:8080/items

With StripBasePath=false:

Input Path: /api/apps/todos/items
UpstreamBaseURL: http://todos-bff.svc:8080

Result: http://todos-bff.svc:8080/api/apps/todos/items

Request & Response Transformation

Based on transformer.go:

Security-First Header Forwarding

Opt-Out Model: Forward all headers by default, then remove blocked ones (lines 95-150).

Default Blocked Headers (defaultBlockedHeaders, lines 23-31):

Cookie                // Session cookies
Set-Cookie // Response cookies
Proxy-Authorization // Proxy auth
Authorization-Legacy // Legacy auth header
X-App-Signature // Internal auth
X-App-Timestamp // Internal auth
X-Real-IP // Will be set by proxy

Default Blocked Patterns (defaultBlockedHeaderPatterns, lines 35-38):

X-Internal-*    // Block all internal headers
X-Forwarded-* // Block all forwarded headers (proxy sets its own)

NOTE: Authorization header is NOT blocked to allow Bearer tokens to pass through.

Header Transformation Pipeline

From TransformHeaders (lines 112-150):

1. Forward all original headers (opt-out model)
2. Remove default blocked headers FIRST (security - lines 128-132)
3. Remove route-configured blocked headers (lines 135-139)
4. Add/override custom headers from route config (lines 142-146)
5. Add standard proxy headers (line 149)

Standard Proxy Headers

Added by addStandardHeaders (lines 152-180):

X-Forwarded-For: <client-ip>[, <proxy-chain>]
X-Forwarded-Proto: https | http
X-Forwarded-Host: <original-host>
X-Real-IP: <client-ip>
X-App-Name: <app-name>
X-App-ID: <app-uuid>

Header Transformation Example

Input request:

GET /api/todos/items HTTP/1.1
Host: client.example.com
Original-Header: value
Cookie: session=xyz # Blocked
X-Internal-Id: secret # Blocked (matches pattern)
Authorization: Bearer xyz # Allowed (needed for auth)

Output to upstream:

GET /items HTTP/1.1
Host: todos-bff.svc:8080
Original-Header: value
Authorization: Bearer xyz
X-Forwarded-For: 192.0.2.1
X-Forwarded-Proto: https
X-Forwarded-Host: client.example.com
X-Real-IP: 192.0.2.1
X-App-Name: todos-app
X-App-ID: 550e8400-e29b-41d4-a716-446655440000

Permission Verification

Based on pkg/v2/infrastructure/middleware/permission.go:

Permission Check Pipeline

From AsHTTPHandler (lines 51-93):

StepFunctionDescription
1Route matchingregistry.Match(path, method)
2Public route checkIf IsPublic=true → allow, skip auth
3Authentication checkRequire auth context for protected routes
4Permission verificationOpenFGA check via multi-level cache
5Response403 if denied, continue if allowed

Authorization Model

OpenFGA Tuple:

Subject: app:{appName} or user:{userID}
Relation: accessible_by
Object: route:{routeID}

Multi-Level Cache Strategy

Level 1: Local Cache (in-memory)
├─ Fastest
├─ 1-second TTL
└─> Shared nothing, per-instance

Level 2: Redis Cache (distributed)
├─ Survives restarts
├─ 5-minute TTL
└─> Shared across instances

Level 3: OpenFGA (source of truth)
├─ Slowest
├─ Always authoritative
└─> Direct API query

Fail Closed: If all caches fail, deny access.

Permission Flow

From permission.go:51-93:

// 1. Match route
matchedRoute, spec, err := m.routeRegistry.Match(req.Context(), req.URL.Path, method)

// 2. Public route - no auth needed
if matchedRoute.IsPublic || spec.IsPublic {
return next(ctx)
}

// 3. Protected route - auth required
authCtx, err := GetAuthContext(req)
if err != nil {
WriteUnauthorizedError(w, "Authentication required")
return nil
}

// 4. Check permission via cache
allowed, err := m.cache.CheckRouteAccess(req.Context(), authCtx, matchedRoute.ID.String())
if !allowed {
WriteForbiddenError(w, "Access denied")
return nil
}

Rate Limiting

Based on pkg/v2/infrastructure/ratelimit/middleware.go:

Hierarchical Rate Limiting

Configuration Priority (lines 43-60):

RouteSpec.RateLimit (highest priority)

RegisteredRoute.RateLimit (fallback)

No limit (if neither configured)

Rate Limit Config:

type RateLimit struct {
RPM int // Requests per minute
Burst int // Burst capacity (must be >= RPM)
}

Middleware Flow

StepFunctionFile:LinesDescription
1Route matchingmiddleware.go:48-54Get RegisteredRoute and RouteSpec
2Determine effective limitmiddleware.go:56-65RouteSpec → Route → None
3Route-level checkmiddleware.go:67-75Key: route:{routeID}, applies to all users
4Per-user checkmiddleware.go:77-91Key: {subject}:route:{routeID}, if authenticated

Rate Limit Response

Headers:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
X-RateLimit-Limit: 60
X-RateLimit-Scope: route | user

Body:

{
"error": {
"code": "RATE_LIMIT_EXCEEDED",
"message": "Route rate limit exceeded"
}
}

Implementation

From middleware.go:67-91:

// Route-level limit (always enforced)
routeKey := fmt.Sprintf("route:%s", registeredRoute.ID.String())
allowed, err := m.limiter.Allow(r.Context(), routeKey)
if !allowed {
w.Header().Set("X-RateLimit-Scope", "route")
w.WriteHeader(http.StatusTooManyRequests)
return
}

// Per-user limit (if authenticated)
if authCtx, err := middleware.GetAuthContext(r); err == nil {
subjectStr, _ := middleware.GetSubjectString(r)
userKey := fmt.Sprintf("%s:route:%s", subjectStr, routeID)
userAllowed, _ := m.limiter.Allow(r.Context(), userKey)
if !userAllowed {
w.Header().Set("X-RateLimit-Scope", "user")
w.WriteHeader(http.StatusTooManyRequests)
return
}
}

Circuit Breaking

Based on pkg/v2/infrastructure/circuitbreaker/:

Per-Upstream Circuit Breaker

Manager Pattern (manager.go):

CircuitBreakerManager
├─ Upstream 1 (http://todos-bff.svc:8080) → CircuitBreaker
├─ Upstream 2 (http://analytics.svc:8080) → CircuitBreaker
└─ Upstream N → CircuitBreaker (lazy creation)

Circuit Breaker States

Closed (Normal)
├─ All requests pass through
├─ Track failure count
└─> Open if threshold exceeded

Open (Fast Fail)
├─ Reject new requests immediately
├─ Return ErrCircuitOpen
└─> Half-Open after timeout

Half-Open (Testing)
├─ Allow limited test requests
├─ Track success count
└─> Closed if success threshold met

Configuration

From proxy_service.go:30-36:

breakerConfig := &circuitbreaker.Config{
FailureThreshold: 5, // Consecutive failures to open
SuccessThreshold: 2, // Consecutive successes to close
Timeout: 60 * time.Second, // Before Half-Open
HalfOpenRequests: 3, // Concurrent test requests
}

Hybrid Approach

From proxy_service.go:99-115:

err = s.breakerManager.Execute(reqCtx, upstreamURL, func() error {
var execErr error
resp, execErr = s.client.Do(upstreamReq)
if execErr != nil {
return execErr // Network error → circuit breaker records failure
}
// Hybrid: 5xx treated as failure for CB but response returned to caller
if resp.StatusCode >= 500 {
return &ProxyError{
StatusCode: resp.StatusCode,
Code: "UPSTREAM_ERROR",
Message: "Upstream service returned error status",
}
}
return nil
})

Benefits:

  • Protects against cascading failures from network errors
  • Allows caller to handle 5xx responses (preserves error details)
  • Fast fail on circuit open (no waiting for timeout)
  • Automatic recovery testing (half-open state)

Failure Classification:

  • Network errors: Circuit breaker blocks with ErrCircuitOpen
  • 5xx responses: Recorded as failures, but response passed to caller
  • <5xx responses: Recorded as success

HTTP Client Configuration

Based on client.go:

Optimized for High Throughput

From NewHTTPClient (lines 15-50):

Transport Settings:
├─ MaxIdleConns: 100 // Total idle connections
├─ MaxIdleConnsPerHost: 10 // Idle per upstream
├─ MaxConnsPerHost: 100 // Max concurrent per upstream
├─ IdleConnTimeout: 90s // Keep-alive duration

Timeouts:
├─ DialContext: 10s // Connection establishment
├─ TLSHandshakeTimeout: 10s // TLS handshake
├─ ResponseHeaderTimeout: 30s // Wait for response headers
├─ ExpectContinueTimeout: 1s // 100-continue

Protocols:
├─ HTTP/2: Enabled (ForceAttemptHTTP2)
├─ Keepalive: 30s
└─ Disable compression: false (gzip enabled)

Error Handling & Status Mapping

Based on error_handler.go:

Error Classification

From HandleError (lines 20-80):

Error TypeHTTP StatusError CodeDescription
ErrCircuitOpen503CIRCUIT_OPENCircuit breaker protecting upstream
DeadlineExceeded504UPSTREAM_TIMEOUTUpstream didn't respond in time
Canceled408REQUEST_CANCELEDClient canceled request
Network Error502CONNECTION_ERRORFailed to connect to upstream
Other502UPSTREAM_ERRORGeneric upstream error

5xx Error Response Wrapping

When upstream returns 5xx (proxy_service.go:240-270):

  1. Read upstream response body
  2. Wrap in standard error envelope
  3. Return original 5xx status code
  4. Include upstream response in details field

Example:

{
"error": {
"code": "UPSTREAM_ERROR",
"message": "Upstream service returned an error",
"details": {
"upstream_status": 500,
"upstream_body": "{\"error\":\"Database connection failed\"}"
}
}
}

Streaming Support

Based on proxy_service.go:

Streaming Detection

From ProxyHTTP (lines 170-300):

Streaming triggers:

  • multipart/form-data (file uploads)
  • Transfer-Encoding: chunked
  • Large request/response bodies

Implementation:

// Request body passed directly (not buffered)
upstreamReq.Body = r.Body

// Response streamed with io.Copy
io.Copy(w, resp.Body)

Preserved Headers

Content-Type (includes boundary for multipart)
Content-Length (if present)
Transfer-Encoding (chunked, etc.)
Content-Encoding (gzip, deflate, br)

Middleware Pipeline

Based on pkg/v2/server/http.go:

Global Middleware Stack

Applied to all routes (lines 200-220):

Request Enters

[1] Logging Middleware
├─ Logs method, path, headers
└─> Continue

[2] Recovery Middleware
├─ Catches panics
└─> Continue

[3] CORS Middleware
├─ Adds CORS headers
└─> Continue

[4] Authentication Middleware (Optional Mode)
├─ Tries to extract auth context
├─ Sets context if found
├─ Doesn't block if missing
└─> Continue

Route Handler

API Proxy Route Stack

Specific to /api/apps/** (lines 289-297):

Request for /api/apps/todos/items

[1] Permission Middleware
├─ Match route in registry
├─ Check if public (skip if yes)
├─ Check authentication (required if not public)
├─ Check route access via OpenFGA
└─> 403 if denied, continue if allowed

[2] Rate Limit Middleware
├─ Check route-level limit
├─ Check per-user limit (if authenticated)
└─> 429 if exceeded, continue if allowed

[3] Proxy Handler (ProxyHTTP)
├─ Transform request
├─ Execute via circuit breaker
├─ Transform response
└─> Return response to client

Telemetry & Observability

Distributed Tracing

From proxy_service.go:54-56, 69-76, 145-146:

Span Attributes:

Span Name: "proxy.ProxyRequest" | "proxy.ProxyHTTP"

Attributes:
├─ route.id: <route-uuid>
├─ route.path: <requested-path>
├─ route.method: <http-method>
├─ app.name: <app-name>
├─ upstream.url: <constructed-url>
├─ http.status_code: <response-status>
└─ request.duration_seconds: <elapsed-time>

Metrics

From proxy_service.go:148-165:

Counters:

api_requests_total
├─ app_name: <app-name>
├─ route: <request-path>
├─ method: <http-method>
└─ status_code: <response-status>

upstream_requests_total
├─ app_name: <app-name>
└─ status_code: <response-status>

Histograms:

api_request_duration_seconds
├─ app_name: <app-name>
├─ route: <request-path>
└─ method: <http-method>

Complete Request Flow

Client Request (GET /api/apps/todos/items?limit=10)

[HTTP Server] Receive on port 8080

[Logging MW] Log request details

[Recovery MW] Panic protection

[CORS MW] Add CORS headers

[Auth MW] Extract auth context (optional)

[Permission MW] OpenFGA check
├─ Route match → FindByPath(/api/apps/todos/items, GET)
├─ Public route? → Continue
├─ No auth context? → 401 Unauthorized
├─ Permission denied? → 403 Forbidden
└─ Allowed → Continue

[Rate Limit MW] Token bucket check
├─ Route limit exceeded? → 429 Too Many Requests
├─ Per-user limit exceeded? → 429 Too Many Requests
└─ Allowed → Continue

[Proxy Handler] ProxyHTTP()
├─ Match route in registry
├─ Build upstream URL: http://todos-bff.svc:8080/items
├─ Transform request headers (security rules)
├─ Add proxy headers (X-Forwarded-*, X-App-*)
├─ Apply timeout (RouteSpec → Route → 30s default)
└─ Execute via circuit breaker
├─ Circuit Open? → 503 Service Unavailable
└─ Circuit Closed → Continue

[HTTP Client] Send to upstream
├─ Connection pooling (reuse existing)
├─ HTTP/2 if supported
└─ Timeout monitoring

[Response] From upstream
├─ 5xx? → Wrap in error envelope
├─ \<5xx? → Stream as-is
├─ Network error? → 502 Bad Gateway
└─ Timeout? → 504 Gateway Timeout

[Metrics] Record counters, histograms

[Tracing] Add span attributes

Response Written to Client

Concurrency Patterns

Thread Safety

Route Registry (registry_inmem.go:11-17):

type inMemoryRegistry struct {
mu sync.RWMutex // Read-write mutex
routes map[uuid.UUID]*RegisteredRoute
}
  • RLock() for concurrent route lookups (hot path - every request)
  • Lock() for registration/deregistration (rare operations)

Circuit Breaker Manager:

  • Per-upstream breaker isolation
  • Thread-safe state transitions
  • Atomic counter operations

Stateless Design:

  • Proxy service has no mutable state
  • All state in external systems (registry, cache, breakers)
  • Enables horizontal scaling

Security Features

  1. Header Security:

    • Default blocked headers (Cookie, X-Internal-*, etc.)
    • Opt-out model prevents accidental exposure
    • Proxy-injected headers cannot be spoofed
  2. Permission Isolation:

    • Per-route OpenFGA authorization
    • Multi-level caching for performance
    • Fail-closed on permission check failures
  3. Rate Limiting:

    • Route-level protection against abuse
    • Per-user fairness enforcement
    • Redis-backed for distributed coordination
  4. Circuit Breaking:

    • Per-upstream isolation prevents cascade failures
    • Fast-fail reduces resource exhaustion
    • Automatic recovery testing
  5. Input Validation:

    • Path injection prevention
    • Header validation and sanitization
    • Body size limits (via HTTP client config)

Performance Characteristics

Latency

Typical Request:

  • Route matching: <1ms (in-memory)
  • Permission check: 1-5ms (L1 cache hit)
  • Rate limit check: 1-3ms (Redis)
  • Upstream request: Variable (depends on service)
  • Total overhead: 2-10ms typical

Throughput

Connection Pooling:

  • 100 max idle connections
  • 10 idle connections per upstream
  • 100 max connections per upstream
  • Efficient connection reuse

HTTP/2:

  • Multiplexing support
  • Header compression
  • Server push capability

Code Reference Table

ComponentFileLinesTests/VerificationDescription
Service Interfaceproxy/service.go9-15integration/proxy_e2e_appserver_test.go:39-120Proxy service contract
ProxyServiceproxy/proxy_service.go15-23Full proxy flow testMain service implementation
ProxyRequestproxy/proxy_service.go49-170Buffered proxy testBuffered request proxy
ProxyHTTPproxy/proxy_service.go172-300Streaming proxy testHTTP streaming proxy
Transformerproxy/transformer.go12-18Header transformation testRequest/response transformation
TransformHeadersproxy/transformer.go112-150Security filter testHeader security rules
defaultBlockedHeadersproxy/transformer.go23-31Unit testedSecurity header blacklist
ErrorHandlerproxy/error_handler.go10-80Error mapping testStatus code mapping
HTTP Clientproxy/client.go15-50Integration testedConnection pooling config
Route Registrydomain/route/registry_inmem.go11-154integration/route_registry_test.goThread-safe route matching
Circuit Breaker Managerinfrastructure/circuitbreaker/manager.go15-120circuitbreaker/manager_test.goPer-upstream breakers
Permission Middlewareinfrastructure/middleware/permission.go15-100integration/authorization_e2e_test.goOpenFGA authorization
Rate Limit Middlewareinfrastructure/ratelimit/middleware.go15-100ratelimit/middleware_test.goHierarchical rate limiting
Proxy Handlerpresentation/http/proxy_handler.go10-60Handler integration testHTTP handler wrapper
Middleware Pipelineserver/http.go220-297Server integration testGlobal + route middleware