Chapter 31: Microservices Architecture

1. Monolith vs Microservices

When a Monolith is RIGHT

2. Decomposition Strategies

Strategy 1: By Business Capability

Advantage: Stable boundaries (business capabilities change slowly). Teams align with business stakeholders.

Strategy 2: By Subdomain (Domain-Driven Design)

Strategy 3: By Verb/Noun

• Noun (Entity Services): Customer Service, Product Service → data ownership, CRUD operations
• Verb (Process Services): Checkout Service, Fraud Detection → orchestrate workflows across entities

Risk: Verb services become orchestration layers with no logic (anemic). Prefer business capability.

Strategy 4: Strangler Fig Pattern (for migration)

graph LR
    A[Client] --> B[API Gateway]
    B -->|New Traffic| C[Microservice 1]
    B -->|New Traffic| D[Microservice 2]
    B -->|Old Traffic| E[Monolith]
    style C fill:#90EE90
    style D fill:#90EE90
    style E fill:#FFB6C1

1. Identify bounded context to extract (e.g., catalog)
2. Build new microservice with same API
3. Route traffic via gateway: 10% → microservice, 90% → monolith
4. Gradually increase percentage, monitor, rollback if issues
5. When 100% on microservice, remove monolith code

Decomposition Decision Diagram

graph TD
    A[Want to extract service?] --> B{High change frequency or scaling need?}
    B -->|No| C[Keep in monolith]
    B -->|Yes| D{Clear business capability boundary?}
    D -->|No| E[Refactor monolith first - extract bounded context]
    D -->|Yes| F{Team can own end-to-end?}
    F -->|No| G[Too fine-grained - combine with related capability]
    F -->|Yes| H{Data dependencies manageable?}
    H -->|No| I[Use shared DB temporarily or event sourcing]
    H -->|Yes| J[Extract as microservice]

3. Bounded Contexts (DDD)

Bounded Context: An explicit boundary within which a domain model is consistent. Outside the boundary, different models apply.

Context Map Patterns

Bounded Context Example

graph TD
    A[Catalog Context] -->|Customer/Supplier| B[Order Context]
    B -->|ACL| C[Legacy ERP]
    A -->|Published Language: ProductAvro| D[Event Bus]
    B -->|Published Language: OrderAvro| D
    E[Inventory Context] -->|Subscribe| D
    F[Shipping Context] -->|Subscribe| D
    G[Payment Context] -->|Open Host Service| B
    style A fill:#ADD8E6
    style B fill:#FFD700
    style E fill:#90EE90
    style F fill:#FFA07A
    style G fill:#DDA0DD

Anti-Corruption Layer Pattern

// Order service internal model
class Order {
  orderId: string
  customerId: string
  items: LineItem[]
  totalAmount: Money
}

// Legacy ERP external model (awful)
class ERPOrder {
  ORD_ID: number
  CUST_NO: string
  ITEMS: string  // comma-separated SKUs
  TOTAL: number  // cents, USD only
}

// ACL translates between models
class ERPAdapter {
  toInternal(erpOrder: ERPOrder): Order {
    return {
      orderId: `ORD-${erpOrder.ORD_ID}`,
      customerId: erpOrder.CUST_NO,
      items: erpOrder.ITEMS.split(',').map(parseSKU),
      totalAmount: Money.cents(erpOrder.TOTAL, 'USD')
    }
  }

  toExternal(order: Order): ERPOrder {
    // reverse translation
  }
}

4. Communication Patterns

Sync vs Async Decision Table

Communication Anti-Pattern: Sync Chain

sequenceDiagram
    participant Client
    participant API Gateway
    participant Order Service
    participant Inventory Service
    participant Payment Service
    participant Shipping Service
    Client->>API Gateway: POST /orders
    API Gateway->>Order Service: createOrder()
    Order Service->>Inventory Service: reserveStock() [200ms]
    Inventory Service-->>Order Service: OK
    Order Service->>Payment Service: charge() [300ms]
    Payment Service-->>Order Service: OK
    Order Service->>Shipping Service: calculateRate() [150ms]
    Shipping Service-->>Order Service: OK
    Order Service-->>API Gateway: Order Created
    API Gateway-->>Client: 201 [650ms total]

Problem: 650ms latency, one service failure fails entire request. Fix: Async choreography or saga orchestration.

5. API Design

REST Best Practices

gRPC for Internal Communication

When NOT gRPC: Browser clients (need gRPC-Web proxy), public APIs (REST more accessible), debugging (binary is hard to inspect).

GraphQL for Aggregation

// Client requests exactly what it needs
query {
  order(id: "123") {
    id
    status
    customer { name, email }
    items { product { name, price }, quantity }
    shipping { carrier, trackingNumber }
  }
}

Benefit: One request aggregates order + customer + product + shipping (no over-fetching, no under-fetching).

Architecture: GraphQL gateway queries order-service, customer-service, product-service, shipping-service via REST/gRPC, stitches response.

Risk: N+1 queries (fetching 100 orders → 100 calls to customer-service). Fix: DataLoader batching.

API Versioning Strategy

Deployment: Run v1 and v2 side-by-side. Gateway routes by header/URL. Sunset v1 after 6 months.

6. Data Management

Database Per Service Pattern

Shared Database Anti-Pattern

graph TD
    A[Order Service] --> B[(Shared DB)]
    C[Inventory Service] --> B
    D[Payment Service] --> B
    E[Shipping Service] --> B
    style B fill:#FF6B6B

Problem:

• Schema changes require coordinating 4 services
• No deployment independence (breaking schema change breaks all)
• DB becomes scaling bottleneck
• Tight coupling defeats microservices benefits

Exception: Read-only replica for analytics/reporting (not operational writes).

Data Duplication Patterns

Handling Joins Across Services

// BAD: JOIN across services (impossible without shared DB)
SELECT o.id, c.name, p.name
FROM orders o
JOIN customers c ON o.customer_id = c.id
JOIN products p ON o.product_id = p.id

// GOOD: API Composition (sync)
orderService.getOrder(123) → { customerId: 456, productId: 789 }
customerService.getCustomer(456) → { name: "Alice" }
productService.getProduct(789) → { name: "Widget" }
// Stitch in API Gateway or BFF

// BETTER: Denormalize (async)
// Order service stores { customerId, customerName, productId, productName }
// When Customer changes name, publish CustomerUpdated event
// Order service subscribes, updates denormalized customerName
// Trade: stale data (eventual consistency) for query simplicity

7. Distributed Transactions

Saga Choreography

sequenceDiagram
    participant Order
    participant Inventory
    participant Payment
    participant Shipping
    Order->>Order: Create order (pending)
    Order->>EventBus: OrderCreated
    EventBus->>Inventory: OrderCreated
    Inventory->>Inventory: Reserve stock
    Inventory->>EventBus: StockReserved
    EventBus->>Payment: StockReserved
    Payment->>Payment: Charge card
    Payment->>EventBus: PaymentCompleted
    EventBus->>Shipping: PaymentCompleted
    Shipping->>Shipping: Schedule shipment
    Shipping->>EventBus: ShipmentScheduled
    EventBus->>Order: ShipmentScheduled
    Order->>Order: Mark order confirmed

Compensation Flow (Payment Fails):

sequenceDiagram
    participant Order
    participant Inventory
    participant Payment
    Payment->>Payment: Charge card (FAIL)
    Payment->>EventBus: PaymentFailed
    EventBus->>Inventory: PaymentFailed
    Inventory->>Inventory: Release stock
    Inventory->>EventBus: StockReleased
    EventBus->>Order: PaymentFailed
    Order->>Order: Mark order cancelled

Saga Orchestration

graph TD
    A[Order Saga Orchestrator] -->|1. Reserve stock| B[Inventory Service]
    B -->|Success| A
    A -->|2. Charge payment| C[Payment Service]
    C -->|Success| A
    A -->|3. Create shipment| D[Shipping Service]
    D -->|Success| A
    A -->|4. Confirm order| E[Order Service]
    C -->|Failure| A
    A -->|Compensate: Release stock| B
    A -->|Compensate: Cancel order| E

Choreography vs Orchestration

Idempotency for At-Least-Once Delivery

// Problem: Kafka delivers StockReserved event twice
// Without idempotency, inventory decrements twice

// Solution: Idempotency key (event ID)
class InventoryService {
  private processedEvents = new Set()

  onStockReserved(event: StockReserved) {
    if (processedEvents.has(event.eventId)) {
      return  // already processed, skip
    }

    // Decrement stock in DB transaction
    db.transaction(() => {
      db.decrementStock(event.sku, event.quantity)
      db.insertProcessedEvent(event.eventId)  // atomic with business logic
    })

    processedEvents.add(event.eventId)
  }
}

8. Service Mesh

What a Service Mesh Provides

Sidecar Proxy Model

graph LR
    A[Service A] -->|localhost| B[Envoy Proxy A]
    B -->|mTLS| C[Envoy Proxy B]
    C -->|localhost| D[Service B]
    E[Control Plane: Istiod] -.->|config| B
    E -.->|config| C

How It Works:

1. Every pod gets an Envoy sidecar proxy injected
2. Service A sends request to localhost:80 (sidecar intercepts via iptables)
3. Sidecar A establishes mTLS connection to Sidecar B, applies retries/timeouts/circuit breaker
4. Sidecar B forwards to Service B on localhost
5. Control plane (Istiod) pushes traffic rules, mTLS certs to sidecars

Service Mesh Configuration Example (Istio)

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: order-service
spec:
  hosts:
  - order-service
  http:
  - match:
    - headers:
        version:
          exact: v2
    route:
    - destination:
        host: order-service
        subset: v2
      weight: 10
  - route:
    - destination:
        host: order-service
        subset: v1
      weight: 90

---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: order-service
spec:
  host: order-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s

Effect: 10% of traffic with header "version: v2" goes to v2 pods. Circuit breaker ejects pod after 5 consecutive 5xx errors.

When NOT to Use Service Mesh

• Small deployments (<10 services): Overhead (CPU, memory, complexity) exceeds benefit
• Non-Kubernetes: Service mesh tooling tightly coupled to K8s
• Performance-critical: Sidecar adds 1-5ms latency per hop

9. Testing Strategies

Testing Pyramid for Microservices

graph TD
    A[End-to-End Tests: 5%] --> B[Integration Tests: 20%]
    B --> C[Contract Tests: 25%]
    C --> D[Unit Tests: 50%]
    style A fill:#FF6B6B
    style B fill:#FFA07A
    style C fill:#FFD700
    style D fill:#90EE90

Contract Testing with Pact

// Consumer: Order Service expects Customer API contract
describe('Customer API', () => {
  const provider = new Pact({ consumer: 'order-service', provider: 'customer-service' })

  it('returns customer by ID', async () => {
    await provider.addInteraction({
      state: 'customer 123 exists',
      uponReceiving: 'a request for customer 123',
      withRequest: { method: 'GET', path: '/customers/123' },
      willRespondWith: {
        status: 200,
        body: { id: '123', name: 'Alice', email: 'alice@example.com' }
      }
    })

    const customer = await orderService.getCustomer('123')
    expect(customer.email).toBe('alice@example.com')
  })
})

// Provider: Customer Service verifies it satisfies contract
describe('Customer Service Pact', () => {
  it('satisfies order-service contract', async () => {
    await pact.verifyProvider({
      provider: 'customer-service',
      pactUrls: ['http://pact-broker/order-service/customer-service/latest']
    })
  })
})

Benefit: If Customer service removes "email" field, provider test fails, blocking deployment before breaking Order service.

Test Data Management

10. Anti-Patterns

Distributed Monolith Example

graph LR
    A[Order Service v1.2] -->|Requires| B[Inventory Service v1.2]
    B -->|Requires| C[Payment Service v1.2]
    C -->|Requires| D[Shipping Service v1.2]
    style A fill:#FF6B6B
    style B fill:#FF6B6B
    style C fill:#FF6B6B
    style D fill:#FF6B6B

Problem: Services are separate, but API changes in Inventory v1.3 break Order. Must coordinate deployment of all 4 services. Fix: Use API versioning so Order v1.2 can call Inventory v1.3 (backward compatible).

11. Migration Strategy

Strangler Fig Pattern (Step-by-Step)

graph TD
    A[Identify Bounded Context to Extract] --> B[Product Catalog: high read load, independent lifecycle]
    B --> C[Build New Microservice with Same API]
    C --> D[Deploy Side-by-Side with Monolith]
    D --> E[Route 10% Traffic via API Gateway]
    E --> F{Monitor Errors, Latency}
    F -->|Issues| G[Rollback to 0%, Fix Microservice]
    F -->|Success| H[Increase to 50%, then 100%]
    H --> I[Remove Code from Monolith]
    I --> J[Repeat for Next Context]

Phase 1: Preparation

Phase 2: Extract First Service

Phase 3: Handle Dependencies

Branch by Abstraction (Alternative)

1. Introduce abstraction layer in monolith: ProductRepository interface
2. Implement MonolithProductRepository (queries local DB)
3. Implement MicroserviceProductRepository (calls product-service API)
4. Use feature flag to toggle: 10% use microservice, 90% use monolith
5. Increase flag percentage, remove monolith implementation

Benefit: Gradual migration without API gateway changes. Risk: Two code paths to maintain during migration.

Parallel Run (for High-Risk Services)

sequenceDiagram
    participant Client
    participant API Gateway
    participant Monolith
    participant Payment Service
    Client->>API Gateway: POST /payments
    API Gateway->>Monolith: charge(card)
    API Gateway->>Payment Service: charge(card) [shadow mode]
    Monolith-->>API Gateway: Success
    Payment Service-->>API Gateway: Success [logged, not returned]
    API Gateway-->>Client: Success (from monolith)

Use Case: Payment processing too risky to canary. Run microservice in shadow mode, log/compare results, don't serve responses. After 1 week of identical results, switch to live traffic.

API Pagination Strategies

Offset Pagination Performance Degradation

Problem: Database must scan and discard all offset rows before returning results. At 1M offset, DB reads 1M rows from disk, then throws them away. No amount of indexing fixes this—it's inherent to OFFSET.

Cursor-Based Pagination (Twitter/Facebook Graph API)

-- Initial request
SELECT * FROM posts WHERE user_id = 123 ORDER BY id DESC LIMIT 20
→ Returns posts [1000, 999, 998, ..., 981]

-- Next page (client sends last_seen_id = 981)
SELECT * FROM posts WHERE user_id = 123 AND id < 981 ORDER BY id DESC LIMIT 20
→ Returns posts [980, 979, ..., 961]

-- Previous page (client sends first_seen_id = 961)
SELECT * FROM posts WHERE user_id = 123 AND id > 961 ORDER BY id ASC LIMIT 20
→ Returns posts [962, 963, ..., 981], reverse order in client

Performance: O(1) with index on (user_id, id). DB seeks directly to id=981 via index, no scanning.

Benefit: Stable under inserts. If 10 new posts added, page 2 still shows same results (id < 981).

Limitation: Can't jump to page 50. Only next/prev. Fine for infinite scroll, bad for "Go to page N" UIs.

Keyset Pagination (Sort by Any Column)

-- Sort by created_at, paginate with composite cursor
SELECT * FROM orders
WHERE (created_at, id) > ('2026-05-01 10:00:00', 12345)
ORDER BY created_at, id
LIMIT 20

-- Response includes cursor for next page
{
  "orders": [...],
  "next_cursor": {
    "created_at": "2026-05-01 10:15:00",
    "id": 12365
  }
}

Why composite cursor? created_at alone isn't unique. Two orders at same timestamp need secondary sort (id) for stable pagination.

Index required: CREATE INDEX idx_orders_created ON orders(created_at, id)

Benefit: Works with any sort column (price, name, etc.), still O(1) performance.

Comparison Table

Real-World Example: Twitter Timeline

GET /api/tweets?user_id=123&limit=20
→ {
  "tweets": [...],
  "next_cursor": "1234567890"  // max_id of last tweet
}

GET /api/tweets?user_id=123&limit=20&cursor=1234567890
→ Returns tweets with id < 1234567890

-- SQL query
SELECT * FROM tweets
WHERE user_id = 123 AND id < 1234567890
ORDER BY id DESC
LIMIT 20

Why Twitter uses cursor: Timeline has millions of tweets. OFFSET 100000 would scan 100K rows. Cursor pagination with (user_id, id) index is O(1) regardless of position.

Handling Deletes and Consistency

API Response Format (Cursor-Based)

// REST API with cursor pagination
GET /api/orders?limit=20

Response:
{
  "data": [
    {"id": 1000, "total": 50.00, "created_at": "2026-05-12T10:00:00Z"},
    {"id": 999, "total": 75.50, "created_at": "2026-05-12T09:45:00Z"},
    ...
  ],
  "pagination": {
    "next_cursor": "eyJjcmVhdGVkX2F0IjogIjIwMjYtMDUtMTJUMDk6MDA6MDBaIiwgImlkIjogOTgxfQ==",
    "has_more": true
  }
}

// Client decodes Base64 cursor → {"created_at": "2026-05-12T09:00:00Z", "id": 981}
GET /api/orders?limit=20&cursor=eyJjcmVhdGVkX2F0IjogIjIwMjYtMDUtMTJUMDk6MDA6MDBaIiwgImlkIjogOTgxfQ==

Why Base64 encode cursor? Hide implementation details (client shouldn't parse it). Allows changing cursor format without breaking clients.