API Gateway Internals: Routing, Authentication, Throttling, Request Transformation & Plugin Architecture

Why Understanding API Gateway Internals Matters

An API gateway is the single entry point for all client requests to your backend services. It handles cross-cutting concerns — authentication, rate limiting, request routing, protocol translation, response transformation — so individual services don't have to. Without a gateway, every service independently implements auth checking, CORS headers, request validation, and logging. With 50 services, that's 50 copies of the same middleware, each with its own bugs and inconsistencies. Understanding how gateways work internally reveals why they're necessary, how to configure them correctly, and when they become bottlenecks.

Without API Gateway:                With API Gateway:
┌────────┐  Auth?                   ┌────────┐
│ Client ├──Rate limit?──►Svc A     │ Client │
│        ├──Auth?───────►Svc B      │        │
│        ├──Auth?───────►Svc C      └───┬────┘
└────────┘  Every service                │
            does everything.        ┌────▼─────────────────────┐
                                    │    API Gateway            │
                                    │ • Auth (once)             │
                                    │ • Rate Limiting (central) │
                                    │ • Routing (path→service)  │
                                    │ • Logging (unified)       │
                                    │ • CORS (one place)        │
                                    └┬────────┬────────┬───────┘
                                     │        │        │
                                    Svc A    Svc B    Svc C
                                    (just    (just    (just
                                     logic)   logic)   logic)

---

API Gateway Architecture

┌──────────────────────────────────────────────────────────────────┐
│ API Gateway Request Pipeline                                     │
│                                                                  │
│  Client Request                                                  │
│       │                                                          │
│       ▼                                                          │
│  ┌──────────────┐                                                │
│  │ TLS Termination│  (decrypt HTTPS → HTTP internally)           │
│  └──────┬───────┘                                                │
│         ▼                                                        │
│  ┌──────────────┐                                                │
│  │ Plugin Chain │  (pre-request plugins run in order)            │
│  │              │                                                │
│  │  ┌─────────────────────────────────────────────┐              │
│  │  │ 1. Request Logger                           │              │
│  │  │ 2. IP Whitelist/Blacklist                   │              │
│  │  │ 3. Rate Limiter                             │              │
│  │  │ 4. Authentication (JWT / API Key / OAuth)   │              │
│  │  │ 5. Authorization (RBAC / scope check)       │              │
│  │  │ 6. Request Validation (schema)              │              │
│  │  │ 7. Request Transformation (header/body)     │              │
│  │  │ 8. Caching (return cached if available)     │              │
│  │  └─────────────────────────────────────────────┘              │
│  └──────┬───────┘                                                │
│         ▼                                                        │
│  ┌──────────────┐                                                │
│  │ Router       │  Match path + method → upstream service        │
│  └──────┬───────┘                                                │
│         ▼                                                        │
│  ┌──────────────┐                                                │
│  │ Load Balancer│  Select backend instance                       │
│  └──────┬───────┘                                                │
│         ▼                                                        │
│  ┌──────────────┐                                                │
│  │ Proxy        │  Forward to upstream, stream response          │
│  └──────┬───────┘                                                │
│         ▼                                                        │
│  ┌──────────────┐                                                │
│  │ Plugin Chain │  (post-response plugins)                       │
│  │              │                                                │
│  │  ┌─────────────────────────────────────────────┐              │
│  │  │ 9. Response Transformation                  │              │
│  │  │ 10. Response Caching (store for next time)  │              │
│  │  │ 11. CORS Headers                            │              │
│  │  │ 12. Response Logger                         │              │
│  │  │ 13. Metrics Collection                      │              │
│  │  └─────────────────────────────────────────────┘              │
│  └──────┬───────┘                                                │
│         ▼                                                        │
│  Client Response                                                 │
└──────────────────────────────────────────────────────────────────┘

---

Building an API Gateway from Scratch

Route Matching

interface RouteConfig {
  id: string;
  method: string;           // GET, POST, * (any)
  path: string;             // /api/users/:id, /api/orders/*
  upstream: UpstreamConfig;
  plugins: PluginConfig[];
  stripPrefix?: string;     // Remove prefix before forwarding
  timeout?: number;         // ms
  retries?: number;
}

interface UpstreamConfig {
  name: string;
  targets: { host: string; port: number; weight: number }[];
  healthCheck?: { path: string; interval: number };
}

interface PluginConfig {
  name: string;
  config: Record<string, any>;
}

// Radix tree router for O(path-length) matching
class RadixRouter {
  private root: RadixNode = { children: new Map(), routes: new Map() };

  addRoute(method: string, path: string, route: RouteConfig): void {
    const segments = this.parsePath(path);
    let current = this.root;
    
    for (const segment of segments) {
      if (!current.children.has(segment)) {
        current.children.set(segment, {
          children: new Map(),
          routes: new Map()
        });
      }
      current = current.children.get(segment)!;
    }
    
    current.routes.set(method.toUpperCase(), route);
  }

  match(method: string, path: string): RouteMatch | null {
    const segments = path.split('/').filter(Boolean);
    const params: Record<string, string> = {};
    
    const result = this.matchRecursive(
      this.root, segments, 0, method.toUpperCase(), params
    );
    
    return result;
  }

  private matchRecursive(
    node: RadixNode,
    segments: string[],
    index: number,
    method: string,
    params: Record<string, string>
  ): RouteMatch | null {
    // Reached end of path
    if (index === segments.length) {
      const route = node.routes.get(method) || node.routes.get('*');
      if (route) return { route, params: { ...params } };
      return null;
    }
    
    const segment = segments[index];
    
    // Exact match
    if (node.children.has(segment)) {
      const result = this.matchRecursive(
        node.children.get(segment)!, segments, index + 1, method, params
      );
      if (result) return result;
    }
    
    // Parameter match (:param)
    for (const [key, child] of node.children) {
      if (key.startsWith(':')) {
        const paramName = key.substring(1);
        params[paramName] = segment;
        const result = this.matchRecursive(
          child, segments, index + 1, method, params
        );
        if (result) {
          return result;
        }
        delete params[paramName];
      }
    }
    
    // Wildcard match (*)
    if (node.children.has('*')) {
      const wildcardNode = node.children.get('*')!;
      const route = wildcardNode.routes.get(method) || wildcardNode.routes.get('*');
      if (route) {
        params['*'] = segments.slice(index).join('/');
        return { route, params: { ...params } };
      }
    }
    
    return null;
  }

  private parsePath(path: string): string[] {
    return path.split('/').filter(Boolean);
  }
}

interface RadixNode {
  children: Map<string, RadixNode>;
  routes: Map<string, RouteConfig>;
}

interface RouteMatch {
  route: RouteConfig;
  params: Record<string, string>;
}

Plugin System

interface GatewayContext {
  request: {
    method: string;
    path: string;
    headers: Record<string, string>;
    query: Record<string, string>;
    body: any;
    clientIp: string;
    params: Record<string, string>;
  };
  response: {
    status: number;
    headers: Record<string, string>;
    body: any;
  };
  route: RouteConfig;
  state: Map<string, any>;  // Shared state across plugins
  startTime: number;
}

type PluginPhase = 'request' | 'response';
type NextFunction = () => Promise<void>;

interface GatewayPlugin {
  name: string;
  phase: PluginPhase;
  priority: number;  // Lower = runs first
  execute(ctx: GatewayContext, config: Record<string, any>, next: NextFunction): Promise<void>;
}

// Authentication Plugin
class AuthPlugin implements GatewayPlugin {
  name = 'auth';
  phase: PluginPhase = 'request';
  priority = 100;

  async execute(ctx: GatewayContext, config: Record<string, any>, next: NextFunction): Promise<void> {
    const authType = config.type || 'jwt';
    
    switch (authType) {
      case 'api-key':
        this.validateApiKey(ctx, config);
        break;
      case 'jwt':
        this.validateJwt(ctx, config);
        break;
      case 'basic':
        this.validateBasic(ctx, config);
        break;
    }
    
    await next();
  }

  private validateApiKey(ctx: GatewayContext, config: Record<string, any>): void {
    const keyHeader = config.headerName || 'x-api-key';
    const apiKey = ctx.request.headers[keyHeader];
    
    if (!apiKey) {
      ctx.response.status = 401;
      ctx.response.body = { error: 'Missing API key' };
      throw new PluginHaltError('auth', 'Missing API key');
    }
    
    // In production: validate against a key store
    const keyData = this.lookupApiKey(apiKey);
    if (!keyData) {
      ctx.response.status = 401;
      ctx.response.body = { error: 'Invalid API key' };
      throw new PluginHaltError('auth', 'Invalid API key');
    }
    
    ctx.state.set('consumer', keyData);
  }

  private validateJwt(ctx: GatewayContext, config: Record<string, any>): void {
    const authHeader = ctx.request.headers['authorization'];
    if (!authHeader?.startsWith('Bearer ')) {
      ctx.response.status = 401;
      ctx.response.body = { error: 'Missing or invalid Authorization header' };
      throw new PluginHaltError('auth', 'Missing token');
    }
    
    const token = authHeader.substring(7);
    
    // Verify JWT signature and expiration
    const payload = this.verifyJwt(token, config.secret);
    if (!payload) {
      ctx.response.status = 401;
      ctx.response.body = { error: 'Invalid or expired token' };
      throw new PluginHaltError('auth', 'Invalid token');
    }
    
    ctx.state.set('user', payload);
    ctx.request.headers['x-user-id'] = payload.sub;
  }

  private validateBasic(ctx: GatewayContext, config: Record<string, any>): void {
    const authHeader = ctx.request.headers['authorization'];
    if (!authHeader?.startsWith('Basic ')) {
      ctx.response.status = 401;
      ctx.response.headers['www-authenticate'] = 'Basic realm="API"';
      ctx.response.body = { error: 'Authentication required' };
      throw new PluginHaltError('auth', 'Missing credentials');
    }
    
    const decoded = Buffer.from(authHeader.substring(6), 'base64').toString();
    const [username, password] = decoded.split(':');
    
    if (!this.validateCredentials(username, password)) {
      ctx.response.status = 401;
      ctx.response.body = { error: 'Invalid credentials' };
      throw new PluginHaltError('auth', 'Invalid credentials');
    }
    
    ctx.state.set('user', { username });
  }

  private lookupApiKey(key: string): any { return { id: '1', plan: 'pro' }; }
  private verifyJwt(token: string, secret: string): any { return { sub: 'user1' }; }
  private validateCredentials(user: string, pass: string): boolean { return true; }
}

class PluginHaltError extends Error {
  constructor(public plugin: string, message: string) {
    super(message);
  }
}

// Request Transformation Plugin
class TransformPlugin implements GatewayPlugin {
  name = 'transform';
  phase: PluginPhase = 'request';
  priority = 200;

  async execute(ctx: GatewayContext, config: Record<string, any>, next: NextFunction): Promise<void> {
    // Add headers
    if (config.addHeaders) {
      for (const [key, value] of Object.entries(config.addHeaders as Record<string, string>)) {
        ctx.request.headers[key] = value;
      }
    }
    
    // Remove headers (e.g., don't forward auth to upstream)
    if (config.removeHeaders) {
      for (const header of config.removeHeaders as string[]) {
        delete ctx.request.headers[header];
      }
    }
    
    // Strip path prefix
    if (ctx.route.stripPrefix) {
      ctx.request.path = ctx.request.path.replace(ctx.route.stripPrefix, '');
      if (!ctx.request.path.startsWith('/')) {
        ctx.request.path = '/' + ctx.request.path;
      }
    }
    
    // Add gateway headers
    ctx.request.headers['x-request-id'] = crypto.randomUUID();
    ctx.request.headers['x-forwarded-for'] = ctx.request.clientIp;
    ctx.request.headers['x-forwarded-proto'] = 'https';
    
    await next();
  }
}

// Response Caching Plugin
class CachePlugin implements GatewayPlugin {
  name = 'cache';
  phase: PluginPhase = 'request';
  priority = 300;
  
  private cache: Map<string, { body: any; headers: Record<string, string>; expiresAt: number }> = new Map();

  async execute(ctx: GatewayContext, config: Record<string, any>, next: NextFunction): Promise<void> {
    // Only cache GET requests
    if (ctx.request.method !== 'GET') {
      await next();
      return;
    }
    
    const cacheKey = `${ctx.request.method}:${ctx.request.path}:${JSON.stringify(ctx.request.query)}`;
    const cached = this.cache.get(cacheKey);
    
    if (cached && cached.expiresAt > Date.now()) {
      ctx.response.status = 200;
      ctx.response.body = cached.body;
      ctx.response.headers = { ...cached.headers, 'x-cache': 'HIT' };
      return;  // Don't call next — return cached response
    }
    
    await next();
    
    // Cache successful responses
    if (ctx.response.status >= 200 && ctx.response.status < 300) {
      const ttl = (config.ttl || 60) * 1000;
      this.cache.set(cacheKey, {
        body: ctx.response.body,
        headers: { ...ctx.response.headers },
        expiresAt: Date.now() + ttl
      });
      ctx.response.headers['x-cache'] = 'MISS';
    }
  }
}

// CORS Plugin
class CorsPlugin implements GatewayPlugin {
  name = 'cors';
  phase: PluginPhase = 'response';
  priority = 900;

  async execute(ctx: GatewayContext, config: Record<string, any>, next: NextFunction): Promise<void> {
    await next();
    
    const allowedOrigins = config.origins || ['*'];
    const origin = ctx.request.headers['origin'];
    
    if (origin && (allowedOrigins.includes('*') || allowedOrigins.includes(origin))) {
      ctx.response.headers['access-control-allow-origin'] = 
        allowedOrigins.includes('*') ? '*' : origin;
      ctx.response.headers['access-control-allow-methods'] = 
        (config.methods || ['GET', 'POST', 'PUT', 'DELETE']).join(', ');
      ctx.response.headers['access-control-allow-headers'] = 
        (config.headers || ['Content-Type', 'Authorization']).join(', ');
      ctx.response.headers['access-control-max-age'] = 
        (config.maxAge || 86400).toString();
    }
  }
}

// Metrics Plugin
class MetricsPlugin implements GatewayPlugin {
  name = 'metrics';
  phase: PluginPhase = 'response';
  priority = 1000;
  
  private counters: Map<string, number> = new Map();
  private latencies: Map<string, number[]> = new Map();

  async execute(ctx: GatewayContext, config: Record<string, any>, next: NextFunction): Promise<void> {
    await next();
    
    const routeId = ctx.route.id;
    const statusClass = `${Math.floor(ctx.response.status / 100)}xx`;
    const key = `${routeId}:${statusClass}`;
    
    this.counters.set(key, (this.counters.get(key) || 0) + 1);
    
    const latency = Date.now() - ctx.startTime;
    if (!this.latencies.has(routeId)) this.latencies.set(routeId, []);
    this.latencies.get(routeId)!.push(latency);
  }
}

Gateway Engine

import * as crypto from 'crypto';

class ApiGateway {
  private router: RadixRouter;
  private plugins: Map<string, GatewayPlugin> = new Map();
  private routes: RouteConfig[] = [];

  constructor() {
    this.router = new RadixRouter();
    
    // Register built-in plugins
    this.registerPlugin(new AuthPlugin());
    this.registerPlugin(new TransformPlugin());
    this.registerPlugin(new CachePlugin());
    this.registerPlugin(new CorsPlugin());
    this.registerPlugin(new MetricsPlugin());
  }

  registerPlugin(plugin: GatewayPlugin): void {
    this.plugins.set(plugin.name, plugin);
  }

  addRoute(config: RouteConfig): void {
    this.routes.push(config);
    this.router.addRoute(config.method, config.path, config);
  }

  async handleRequest(
    method: string,
    path: string,
    headers: Record<string, string>,
    query: Record<string, string>,
    body: any,
    clientIp: string
  ): Promise<{ status: number; headers: Record<string, string>; body: any }> {
    // Route matching
    const match = this.router.match(method, path);
    
    if (!match) {
      return {
        status: 404,
        headers: { 'content-type': 'application/json' },
        body: { error: 'Not Found', message: `No route matches ${method} ${path}` }
      };
    }
    
    // Build context
    const ctx: GatewayContext = {
      request: { method, path, headers, query, body, clientIp, params: match.params },
      response: { status: 200, headers: {}, body: null },
      route: match.route,
      state: new Map(),
      startTime: Date.now()
    };
    
    try {
      // Gather applicable plugins for this route
      const routePlugins = this.getRoutePlugins(match.route);
      
      // Build and execute the plugin chain
      const requestPlugins = routePlugins
        .filter(p => p.plugin.phase === 'request')
        .sort((a, b) => a.plugin.priority - b.plugin.priority);
      
      const responsePlugins = routePlugins
        .filter(p => p.plugin.phase === 'response')
        .sort((a, b) => a.plugin.priority - b.plugin.priority);
      
      // Execute request plugins
      await this.executeChain(ctx, requestPlugins, 0, async () => {
        // Proxy to upstream
        await this.proxyToUpstream(ctx);
      });
      
      // Execute response plugins
      await this.executeChain(ctx, responsePlugins, 0, async () => {});
      
    } catch (error) {
      if (error instanceof PluginHaltError) {
        // Plugin intentionally halted the request (e.g., auth failure)
      } else {
        ctx.response.status = 502;
        ctx.response.body = { error: 'Bad Gateway' };
      }
    }
    
    return {
      status: ctx.response.status,
      headers: ctx.response.headers,
      body: ctx.response.body
    };
  }

  private getRoutePlugins(route: RouteConfig): { plugin: GatewayPlugin; config: Record<string, any> }[] {
    return route.plugins
      .map(pc => {
        const plugin = this.plugins.get(pc.name);
        if (!plugin) return null;
        return { plugin, config: pc.config };
      })
      .filter(Boolean) as { plugin: GatewayPlugin; config: Record<string, any> }[];
  }

  private async executeChain(
    ctx: GatewayContext,
    plugins: { plugin: GatewayPlugin; config: Record<string, any> }[],
    index: number,
    finalHandler: () => Promise<void>
  ): Promise<void> {
    if (index >= plugins.length) {
      await finalHandler();
      return;
    }
    
    const { plugin, config } = plugins[index];
    await plugin.execute(ctx, config, async () => {
      await this.executeChain(ctx, plugins, index + 1, finalHandler);
    });
  }

  private async proxyToUpstream(ctx: GatewayContext): Promise<void> {
    const upstream = ctx.route.upstream;
    
    // Simple round-robin target selection
    const target = upstream.targets[
      Math.floor(Math.random() * upstream.targets.length)
    ];
    
    const timeout = ctx.route.timeout || 30000;
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), timeout);
    
    try {
      const url = `http://${target.host}:${target.port}${ctx.request.path}`;
      
      const response = await fetch(url, {
        method: ctx.request.method,
        headers: ctx.request.headers,
        body: ctx.request.method !== 'GET' ? JSON.stringify(ctx.request.body) : undefined,
        signal: controller.signal
      });
      
      ctx.response.status = response.status;
      ctx.response.body = await response.json().catch(() => response.text());
      
      response.headers.forEach((value, key) => {
        ctx.response.headers[key] = value;
      });
    } catch (error: any) {
      if (error.name === 'AbortError') {
        ctx.response.status = 504;
        ctx.response.body = { error: 'Gateway Timeout' };
      } else {
        ctx.response.status = 502;
        ctx.response.body = { error: 'Bad Gateway', message: 'Upstream unreachable' };
      }
    } finally {
      clearTimeout(timer);
    }
  }
}

// Usage example
function configureGateway(): ApiGateway {
  const gateway = new ApiGateway();
  
  gateway.addRoute({
    id: 'users-api',
    method: '*',
    path: '/api/users/:id',
    upstream: {
      name: 'user-service',
      targets: [
        { host: '10.0.1.10', port: 3000, weight: 1 },
        { host: '10.0.1.11', port: 3000, weight: 1 }
      ]
    },
    stripPrefix: '/api',
    timeout: 10000,
    retries: 2,
    plugins: [
      { name: 'auth', config: { type: 'jwt', secret: 'secret' } },
      { name: 'transform', config: { addHeaders: { 'x-service': 'gateway' } } },
      { name: 'cache', config: { ttl: 60 } },
      { name: 'cors', config: { origins: ['https://app.example.com'] } },
      { name: 'metrics', config: {} }
    ]
  });
  
  return gateway;
}

---

Comparison Table

┌──────────────────┬────────────────┬────────────────┬────────────────┬────────────────┐
│                  │ Kong           │ AWS API GW     │ Envoy          │ Nginx          │
│                  │ (Plugin-based) │ (Managed)      │ (Sidecar/Edge) │ (Reverse Proxy)│
├──────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ Type             │ Standalone GW  │ Fully managed  │ Proxy/mesh     │ Web server +   │
│                  │                │                │                │ reverse proxy  │
│ Plugin ecosystem │ 100+ plugins   │ Lambda-based   │ WASM filters   │ Lua/njs        │
│                  │                │ authorizers    │                │ modules        │
│ Protocol support │ HTTP, gRPC,    │ HTTP, WS,      │ HTTP/1.1, H2,  │ HTTP, TCP,     │
│                  │ TCP, WS        │ REST           │ gRPC, TCP, UDP │ UDP            │
│ Latency overhead │ 1-5ms          │ 10-30ms        │ <1ms           │ <1ms           │
│ Rate limiting    │ Built-in       │ Built-in       │ Via config     │ ngx_limit_req  │
│ Auth             │ JWT, OAuth,    │ Cognito, IAM,  │ JWT, ext auth  │ Basic, JWT     │
│                  │ LDAP, etc.     │ Lambda         │ service        │ via module     │
│ Scaling          │ Horizontal     │ Auto-scale     │ Per-pod side-  │ Horizontal     │
│                  │ with DB        │ (managed)      │ car scaling    │                │
│ Best for         │ Multi-service  │ AWS-native     │ Service mesh   │ Simple routing │
│                  │ API management │ serverless      │ Kubernetes     │ high perf      │
└──────────────────┴────────────────┴────────────────┴────────────────┴────────────────┘

---

Interview Questions

Q1: What's the difference between an API gateway and a reverse proxy?

A reverse proxy (Nginx, HAProxy) forwards requests to backend servers and handles basic concerns: TLS termination, load balancing, static file serving, request buffering. An API gateway is a superset — it's a reverse proxy plus application-level features: authentication/authorization, rate limiting per consumer, request/response transformation, API versioning, developer portal, analytics, and a plugin architecture for custom logic. Key differences: A reverse proxy routes by host/path to a backend. An API gateway routes by host/path/method/headers to a backend AND applies a configurable chain of middleware (plugins) per route. Reverse proxies are infrastructure-level. API gateways are application-level. In practice, many teams use both: Nginx as L7 reverse proxy for TLS and static assets, with Kong/Envoy as the API gateway for API-specific concerns.

Q2: How do you prevent the API gateway from becoming a single point of failure?

(1) Horizontal scaling: Deploy multiple gateway instances behind a load balancer (L4 or DNS-based). Gateways should be stateless — all configuration in a database/config store, all session state in Redis. (2) Health checks: The load balancer actively health-checks gateway instances and removes unhealthy ones. (3) Configuration replication: Kong uses a PostgreSQL/Cassandra backend. Each gateway instance caches configuration in memory and syncs periodically. If the config store is temporarily down, gateways continue operating with cached config. (4) Circuit breaking: If an upstream service is down, the gateway should fail fast (circuit breaker) instead of holding connections. (5) Rate limiting at the gateway level: Protect the gateway itself from overload by limiting total requests/second globally. (6) Graceful degradation: If Redis (rate limiting store) is down, fail open (allow requests) rather than rejecting all traffic. Key principle: The gateway adds latency (1-10ms) and complexity, but the alternative (each service implementing auth/rate-limiting independently) is worse.

Q3: How does the plugin/middleware chain work in API gateways like Kong?

Kong's plugin chain follows the onion model (similar to Express middleware). Each plugin can: (1) Inspect/modify the request, (2) Short-circuit the response (e.g., auth failure → 401 without reaching upstream), (3) Call next() to pass to the next plugin, (4) Inspect/modify the response after upstream replies. Execution order: Plugins run by priority (lower number = earlier). Request phases: certificate → rewrite → access → header_filter → body_filter → log. The access phase is where auth and rate limiting happen. If auth fails, subsequent plugins don't run. Per-route configuration: Each route declares which plugins are active and their configs. Route A might have JWT auth + rate limiting. Route B might have API key auth + caching. Custom plugins: In Kong, plugins are Lua modules (or Go/Python via plugin servers). In Envoy, WASM filters. In AWS API Gateway, Lambda authorizers.

Q4: How should you handle API versioning at the gateway level?

Three approaches: (1) URL path versioning (/v1/users, /v2/users): The gateway routes /v1/* to service-v1 and /v2/* to service-v2. Simple, explicit, cacheable. Most common in practice. (2) Header versioning (Accept: application/vnd.api.v2+json): The gateway inspects the Accept header and routes to the appropriate backend. Cleaner URLs but harder to test (can't just change the URL). (3) Query parameter (/users?version=2): Least common, pollutes the URL. Gateway implementation: Define separate routes per version, each pointing to a different upstream. Use the gateway's request transformation plugin to strip the version prefix before forwarding (/v2/users → /users to the v2 service). Deprecation: Configure the gateway to add a Sunset header to v1 responses with the deprecation date. After the date, return 410 Gone. Canary routing: Use the gateway to send 10% of v1 traffic to v2 (testing in production) by configuring weighted upstreams.

Q5: What is the performance impact of an API gateway and how do you optimize it?

Typical overhead: 1-10ms per request depending on the gateway and enabled plugins. Kong: ~2-5ms. Envoy: ~0.5-1ms. AWS API Gateway: ~10-30ms. Breakdown: TLS termination: ~1ms (hardware accelerated). Plugin chain execution: ~0.5-3ms (depends on number and complexity of plugins). Upstream proxying: ~0.5-1ms (TCP connection, request forwarding). Optimization: (1) Connection pooling: Keep persistent connections to upstream services (avoid TCP handshake per request). (2) Disable unnecessary plugins: Each plugin adds latency. Only enable what's needed per route. (3) Cache authentication: JWT validation is CPU-intensive (signature verification). Cache validated tokens for their remaining TTL. (4) Response caching: Cache upstream responses at the gateway for read-heavy endpoints. (5) TLS session resumption: Reuse TLS sessions for returning clients (TLS 1.3 0-RTT). (6) Rate limiting with local counters: Use local in-memory counters with periodic Redis sync instead of checking Redis on every request. (7) Binary protocols: Use HTTP/2 or gRPC between gateway and upstreams to reduce overhead.

---

Key Takeaways

1. An API gateway centralizes cross-cutting concerns: Auth, rate limiting, CORS, logging, and metrics are implemented once instead of in every service.

2. The plugin chain follows the onion model: Request plugins run in priority order, then upstream proxying, then response plugins in reverse. Any plugin can short-circuit.

3. Gateways must be stateless and horizontally scalable: Configuration cached locally, session state in Redis, health-checked by a load balancer.

4. Typical overhead is 1-10ms per request: Keep plugins minimal. Cache auth tokens. Use connection pooling to upstream services.

5. Path-based routing with radix trees provides O(path-length) matching: Faster than linear route scanning for large numbers of routes.

6. Request transformation should strip internal headers from responses: Don't leak x-internal-*, server version, or debug information to clients.

7. Circuit breaking at the gateway prevents cascade failures: If an upstream is unresponsive, fail fast rather than exhausting gateway connections.

8. API versioning via URL paths is the most practical approach: /v1/ and /v2/ route to different upstreams. Use the gateway to add Sunset headers for deprecated versions.

9. Kong, Envoy, and AWS API Gateway serve different niches: Kong for multi-cloud API management, Envoy for service mesh, AWS API Gateway for serverless.

10. Monitor gateway metrics religiously: Request rate, error rate, latency percentiles (p50/p95/p99), and upstream health are the critical signals.