Chapter 77: Edge Computing & IoT at Scale

Edge computing brings computation and data storage closer to the devices that generate it. When combined with IoT (Internet of Things), you're managing millions of devices streaming telemetry, processing locally, and syncing with the cloud. This chapter covers the architecture patterns that make it work at scale.

Edge-Cloud Hybrid Architecture

Not everything should run in the cloud. Not everything should run on the device. The art is knowing where to place each workload.

graph TB
    Device[IoT Device
Sensors, Local Logic]
    Edge[Edge Gateway
Aggregation, Filtering]
    Regional[Regional Cloud
Processing, Storage]
    Central[Central Cloud
Analytics, ML Training]

    Device -->|Raw Telemetry| Edge
    Edge -->|Filtered Data| Regional
    Regional -->|Aggregated Metrics| Central
    Central -->|Model Updates| Regional
    Regional -->|Config, Firmware| Edge
    Edge -->|Commands| Device

Decision Criteria: What Runs Where?

Layer Runs Here Why
Device Sensor reading, immediate response (safety), local caching Zero latency, works offline, minimal bandwidth
Edge Gateway Filtering, aggregation, protocol translation, local inference Reduce cloud traffic, batch small events, hide device complexity
Regional Cloud Real-time processing, dashboards, model inference Low latency for users, regional compliance, high compute
Central Cloud ML training, cross-region analytics, historical queries Unlimited scale, centralized management, long-term storage
Interview Tip: When asked about edge vs cloud, always start with latency and bandwidth constraints. Example: "A self-driving car can't wait 100ms for a cloud decision—that's 3 meters at highway speed. So collision avoidance runs on-device. Route planning can run in the cloud."

MQTT Protocol for IoT

MQTT (Message Queuing Telemetry Transport) is the de facto standard for IoT. It's a lightweight pub/sub protocol designed for unreliable networks and low-power devices.

Publish/Subscribe Model

graph LR
    P1[Publisher: Sensor 1]
    P2[Publisher: Sensor 2]
    Broker[MQTT Broker]
    S1[Subscriber: Dashboard]
    S2[Subscriber: Alert Service]
    S3[Subscriber: Storage]

    P1 -->|Publish to topic
'sensors/temp'| Broker
    P2 -->|Publish to topic
'sensors/humidity'| Broker
    Broker -->|Subscribe| S1
    Broker -->|Subscribe| S2
    Broker -->|Subscribe| S3
Topic Hierarchy
home/livingroom/temperature
home/livingroom/humidity
home/bedroom/temperature
factory/line1/machine5/vibration
factory/line1/machine5/temp

# Wildcards
home/+/temperature          # Single level (+)
home/#                      # Multi-level (#)
factory/line1/machine5/#    # All metrics for machine5

Quality of Service (QoS) Levels

QoS Guarantee Use Case Performance
0 At most once (fire and forget) Temperature readings—occasional loss OK Fastest, no acks
1 At least once (may duplicate) Alerts—duplicate OK, loss not OK Fast, single ack
2 Exactly once (4-way handshake) Billing events—no loss or duplicates Slowest, guaranteed
Interview Insight: Most IoT traffic uses QoS 0 (telemetry) or 1 (commands). QoS 2 is rare because the cost is high. If you need exactly-once semantics, consider idempotency tokens instead.

Device Management at Scale

Managing 10 devices is easy. Managing 10 million requires infrastructure.

Device Lifecycle

stateDiagram-v2
    [*] --> Manufactured
    Manufactured --> Provisioned: Claim device
    Provisioned --> Active: First connect
    Active --> Updating: OTA firmware
    Updating --> Active: Success
    Updating --> Failed: Error
    Failed --> Active: Rollback
    Active --> Inactive: Network loss
    Inactive --> Active: Reconnect
    Active --> Retired: End of life
    Retired --> [*]

Key Capabilities

Capability Description Implementation
Provisioning Securely onboard new devices Certificate-based auth, device claims API, zero-touch setup
OTA Updates Push firmware over-the-air Staged rollouts, A/B partitions, rollback on failure, delta updates
Fleet Management Organize devices into groups Tags, hierarchies, dynamic groups (query-based), bulk operations
Remote Control Send commands to devices Shadow state (desired vs reported), command queue, timeout handling
Health Monitoring Track connectivity and issues Heartbeats, last-seen timestamps, error logs, metrics dashboard
Device Shadow Pattern
{
  "deviceId": "sensor-12345",
  "state": {
    "desired": {
      "firmwareVersion": "2.1.0",
      "reportingInterval": 60
    },
    "reported": {
      "firmwareVersion": "2.0.5",
      "reportingInterval": 60,
      "batteryLevel": 87,
      "lastBoot": "2026-05-12T08:30:00Z"
    }
  },
  "metadata": {
    "desired": {
      "firmwareVersion": {
        "timestamp": 1715515200
      }
    }
  }
}

Telemetry Pipeline

Getting data from millions of devices into a queryable data store requires a multi-stage pipeline.

graph LR
    D1[Device] -->|MQTT| GW[Gateway]
    D2[Device] -->|MQTT| GW
    D3[Device] -->|MQTT| GW
    GW -->|Batch| Broker[Message Broker
Kafka/Kinesis]
    Broker -->|Stream| SP[Stream Processor
Flink/Kinesis Analytics]
    SP -->|Hot Data| TS[Time-Series DB
InfluxDB/TimescaleDB]
    SP -->|Alerts| Alert[Alert Service]
    SP -->|Cold Data| S3[Object Storage
S3/GCS]
    S3 -->|Query| Athena[Query Engine
Athena/BigQuery]

Pipeline Stages

  1. Ingestion: Devices publish to MQTT broker or HTTP endpoint
  2. Edge Filtering: Gateway drops noisy data, aggregates readings
  3. Message Broker: Kafka/Kinesis buffers and distributes data
  4. Stream Processing: Real-time transformations, alerting, windowing
  5. Hot Storage: Recent data in time-series DB (last 30 days)
  6. Cold Storage: Historical data in object storage (compressed, partitioned)
Interview Scenario: "10 million smart meters report every 15 minutes. That's 666,666 messages/minute or 11,111/sec. Each message is 200 bytes. Bandwidth = 11,111 × 200 = 2.2 MB/sec. Kafka can handle this easily. But if you want per-device queries, you need partitioning by device_id or region."

Offline-First Architecture

IoT devices lose connectivity. Design for it from day one.

Local Processing

Offline Queue Pattern
class TelemetryQueue:
    def __init__(self):
        self.queue = deque()
        self.max_size = 10000

    def enqueue(self, reading):
        if len(self.queue) >= self.max_size:
            self.queue.popleft()  # Drop oldest
        self.queue.append(reading)

    def flush(self, mqtt_client):
        while self.queue and mqtt_client.is_connected():
            reading = self.queue.popleft()
            try:
                mqtt_client.publish(topic, reading)
            except Exception:
                self.queue.appendleft(reading)  # Retry later
                break

Sync Strategies

Strategy When to Use Trade-offs
Full Sync Small datasets, infrequent changes Simple, but wasteful bandwidth
Delta Sync Large datasets, frequent changes Efficient, but needs version tracking
Last-Write-Wins Single writer per key Simple, but loses concurrent updates
Conflict-Free Replicated Data Types (CRDTs) Multiple writers, no central authority Always converges, but complex

Scalability: Millions of Devices

Connection Management

Each device holds an open connection. At scale, this is your bottleneck.

Connection Limits
- Single MQTT broker: ~100K concurrent connections
- To reach 10M devices: need 100 brokers or clustered setup
- Memory per connection: ~10KB (TLS adds more)
- File descriptors: increase ulimit, use epoll/kqueue

Scalability Patterns

Pattern Description Scale
Broker Clustering Multiple MQTT brokers share load (e.g., HiveMQ, VerneMQ) Millions of connections
Regional Deployment Devices connect to nearest region, reduce latency Global footprint
Connection Throttling Limit reconnects during outages (exponential backoff + jitter) Prevents thundering herd
Message Sampling Only process % of telemetry, aggregate rest Reduce backend load 
graph TB
    subgraph Region1[US-West]
        LB1[Load Balancer]
        B1[MQTT Broker 1]
        B2[MQTT Broker 2]
        LB1 --> B1
        LB1 --> B2
    end

    subgraph Region2[EU-Central]
        LB2[Load Balancer]
        B3[MQTT Broker 3]
        B4[MQTT Broker 4]
        LB2 --> B3
        LB2 --> B4
    end

    Devices1[Devices
US-West] --> LB1
    Devices2[Devices
Europe] --> LB2

    B1 --> Kafka[Global Kafka Cluster]
    B2 --> Kafka
    B3 --> Kafka
    B4 --> Kafka
Staff-Level Signal: Discuss failure modes. "If a broker crashes, can devices reconnect without overloading survivors? We use exponential backoff (1s, 2s, 4s...) plus random jitter to spread reconnections. Also, connection draining—gracefully move connections before maintenance."

Cost Optimization

Real-World Example: Smart Building

Scenario: 500 buildings, 10,000 sensors each (5M total). Sensors report every 60 seconds.

Architecture:

Scaling:

Summary

Final Interview Tip: Always mention scale numbers. "Each broker handles 100K connections. 10M devices = 100 brokers. At $X/broker/month, that's $Y. Edge gateways reduce this to 10K connections = 1 broker. Cost drops 100x." Show you think about real-world constraints.