Chapter 80: Trade-Off Decision Framework

System design has no perfect answers. Every decision is a trade-off. The interviewer isn't testing if you know "the right answer"—they're testing if you can reason about trade-offs, justify your choices, and adapt when constraints change. This chapter gives you a framework to make and defend decisions under pressure.

The STAR Method for Trade-Offs

Borrowed from behavioral interviews, STAR keeps your trade-off discussions structured and complete.

graph LR
    S[State Options] --> T[Trade-offs]
    T --> A[Argue Preference]
    A --> R[Risks & Mitigations]
Step What You Do Example
State Present 2-3 viable options "We could use SQL or NoSQL for the database"
Trade-offs List pros/cons of each "SQL gives ACID but limited horizontal scale. NoSQL scales but eventual consistency."
Argue Pick one and justify for THIS context "For Twitter, I'd use NoSQL because we're read-heavy and can tolerate eventual consistency on timelines."
Risks Acknowledge downsides and how you'd mitigate "Risk: eventual consistency could show stale tweets. Mitigation: use timestamps and show 'loading newer tweets' indicator."
Anti-Pattern: Saying "It depends on the use case" and stopping there. ALWAYS commit to a choice for the given context. The interviewer wants to see decisiveness backed by reasoning.

Common Trade-Offs Table

These pairs appear in every system design interview. Memorize the dimensions.

1. Consistency vs Availability (CAP Theorem)

Dimension Consistency (CP) Availability (AP)
Guarantee All nodes see same data, may reject writes during partition Always accept reads/writes, may return stale data
Use Case Banking, inventory, bookings (correctness critical) Social feeds, recommendations, metrics (staleness OK)
Technology PostgreSQL, MongoDB (strong consistency mode), Zookeeper Cassandra, DynamoDB, Riak
Failure Mode Service becomes read-only or unavailable Writes succeed but replicas diverge temporarily
Interview Example:

Question: "Should we prioritize consistency or availability for a ride-sharing app?"

STAR Answer:

2. Latency vs Throughput

Dimension Latency-Optimized Throughput-Optimized
Definition How fast one request completes How many requests we can handle per second
Strategy Fewer connections, smaller batches, in-memory Larger batches, async processing, disk I/O
Use Case Real-time apps (gaming, video calls, trading) Batch jobs (ETL, log processing, backups)
Example Redis (sub-ms reads), HTTP/2 (multiplexing) Kafka (batch writes), S3 (large uploads)
Real-World: Netflix prioritizes throughput (serve millions of streams) over latency (buffering 2 seconds of video is fine). A stock trading platform prioritizes latency (execute order in <10ms) over throughput (fewer traders, but each needs instant response).

3. Cost vs Performance

Dimension Cost-Optimized Performance-Optimized
Approach Vertical scale, spot instances, cold storage Horizontal scale, on-demand, SSD, in-memory cache
Use Case Startup MVP, internal tools, batch processing User-facing, high-traffic, competitive SLAs
Infrastructure Single region, shared DB, S3 Glacier, fewer replicas Multi-region, sharded DB, Redis cache, 3x replication
Risk Slower response, downtime during spikes High monthly bill, over-provisioning
Interview Insight: Always ask about budget constraints. "Are we optimizing for cost or performance?" If the interviewer says "startup with $10K/month budget," don't design a multi-region Kubernetes cluster. Show pragmatism.

4. Simplicity vs Scalability

Dimension Simple Scalable
Architecture Monolith, single DB, synchronous calls Microservices, sharded DB, async messaging
Operations Easy to deploy, debug, reason about Complex orchestration, distributed tracing
Use Case 0-1M users, small team, MVP 10M+ users, large org, proven product
When to Migrate Start here Migrate when monolith becomes bottleneck
Classic Mistake: Designing microservices for a startup with 100 users. Interviewer: "Why not start with a monolith?" You: "Because microservices scale better." Interviewer: "But you don't have scale yet. Now you have 10x the operational complexity. Why?" Strong answer: "You're right. I'd start with a monolith. Once we hit 1M users and the team grows to 20 engineers, we can extract microservices for hotspots (e.g., payment service)."

5. SQL vs NoSQL

Dimension SQL (Relational) NoSQL
Data Model Structured, fixed schema, ACID transactions Flexible schema, denormalized, BASE (eventual consistency)
Scaling Vertical (limited), read replicas (read-heavy), sharding (complex) Horizontal (easy), partition by key
Queries Complex joins, aggregations, ad-hoc queries Key-value lookups, range scans, limited joins
Use Case Financial transactions, e-commerce orders, user accounts Timelines, logs, session storage, IoT data
Examples PostgreSQL, MySQL Cassandra, DynamoDB, MongoDB
Pro Tip: Don't say "NoSQL is faster." It depends. SQL with proper indexes can be just as fast for relational data. The real advantage of NoSQL is horizontal scaling and schema flexibility, not raw speed.

6. Push vs Pull (Data Delivery)

Dimension Push (Server → Client) Pull (Client → Server)
Mechanism WebSocket, SSE, push notifications Polling (short or long), request-response
Latency Real-time (instant when event occurs) Delayed by polling interval (1-60 sec)
Server Load Constant connections (1 per client) Bursty load (all clients poll at interval)
Use Case Chat, live scores, stock tickers, multiplayer games Email client, news feed, dashboard (refresh every 30s)
Complexity Need persistent connections, state management Stateless, simpler infrastructure

7. Normalization vs Denormalization

Dimension Normalized Denormalized
Structure Data split across tables, no duplication Redundant data, pre-joined
Writes Fast (update one place) Slow (update multiple places)
Reads Slow (joins required) Fast (no joins, direct lookup)
Use Case Write-heavy, OLTP (e.g., order management) Read-heavy, OLAP (e.g., dashboards, timelines)
Example User table, Post table (foreign key user_id) Timeline table with user info embedded in each post

8. Synchronous vs Asynchronous

Dimension Synchronous Asynchronous
Flow Client waits for response (blocking) Client gets immediate ack, processing happens later
Latency Higher perceived latency (user waits) Lower perceived latency (instant feedback)
Reliability Failure = user sees error immediately Failure = retry in background, user notified later
Use Case Payment processing, login, booking Email sending, video encoding, feed fanout
Implementation REST API call, RPC Message queue (Kafka, SQS), background workers

9. Vertical vs Horizontal Scaling

Dimension Vertical (Scale Up) Horizontal (Scale Out)
Approach Add CPU, RAM to existing server Add more servers
Limit Hardware ceiling (e.g., 96 cores, 1TB RAM) Unlimited (add more machines)
Complexity Simple (no code changes) Complex (sharding, load balancing, coordination)
Cost Non-linear (2x resources ≠ 2x cost, often 3-4x) Linear (2x servers = 2x cost)
Use Case Quick wins, small scale (< 1M users) Large scale (10M+ users), need redundancy

10. Batch vs Stream Processing

Dimension Batch Stream
Processing Process large dataset at once (hourly, daily) Process events as they arrive (real-time)
Latency High (wait for batch to complete) Low (sub-second)
Throughput High (optimized for large volumes) Lower (per-event overhead)
Use Case ETL, reporting, model training Fraud detection, real-time dashboards, alerting
Technology Spark, MapReduce, Airflow Flink, Kafka Streams, Storm

How to Present Trade-Offs (Live in Interview)

Template Structure

Trade-Off Discussion Template
1. ACKNOWLEDGE BOTH SIDES
   "There are two main approaches: X and Y."

2. COMPARE (use table or pros/cons)
   "X offers A and B, but sacrifices C.
    Y offers C and D, but sacrifices A."

3. STATE YOUR CHOICE
   "For this use case, I would choose X because..."

4. JUSTIFY WITH CONTEXT
   "Given that we have [constraint/requirement], X is better because..."

5. ACKNOWLEDGE RISKS
   "The downside of X is [risk]. To mitigate, I would [solution]."

6. SHOW FLEXIBILITY
   "If the requirements change (e.g., [scenario]), I would reconsider Y."

Example: Cache Eviction Policy

Scenario: Designing a caching layer. Interviewer asks: "What eviction policy would you use?"

Weak Answer: "I'd use LRU because it's standard."

Strong Answer:

"There are three common eviction policies: LRU (Least Recently Used), LFU (Least Frequently Used), and FIFO. LRU evicts items that haven't been accessed recently. LFU evicts items that are accessed infrequently. FIFO evicts oldest items."

"For a news website, I'd choose LRU because recent news is hot (yesterday's article gets 10K views, last week's gets 100 views). LRU ensures we keep trending content in cache."

"The risk with LRU is 'cache pollution'—a one-time spike (e.g., bot traffic) could evict useful items. To mitigate, I'd use a sliding window LRU or add a frequency counter (LRU-K, which tracks last K accesses)."

"If this were a music app where certain songs are popular year-round (not recency-dependent), I'd reconsider LFU."

Decision Matrices (Weighted Scoring)

When comparing 3+ technologies, a weighted matrix makes your reasoning transparent.

Example: Choosing a Database

Criteria Weight PostgreSQL Cassandra DynamoDB
Horizontal Scalability 30% 6 (sharding complex) 10 (excellent) 10 (excellent)
Query Flexibility 20% 10 (SQL, joins) 5 (limited) 4 (key-value only)
ACID Transactions 25% 10 (full ACID) 5 (lightweight) 7 (conditional writes)
Operational Overhead 15% 7 (replication setup) 5 (cluster management) 10 (fully managed)
Cost 10% 8 (self-hosted) 7 (self-hosted) 6 (pay-per-request)
Weighted Score - 7.95 7.35 8.05
Calculation:
PostgreSQL = 6×0.3 + 10×0.2 + 10×0.25 + 7×0.15 + 8×0.1 = 7.95
Cassandra = 10×0.3 + 5×0.2 + 5×0.25 + 5×0.15 + 7×0.1 = 7.35
DynamoDB = 10×0.3 + 4×0.2 + 7×0.25 + 10×0.15 + 6×0.1 = 8.05

Conclusion: "Given that scalability is our top priority (30% weight) and we have limited ops capacity (15% weight), DynamoDB scores highest. We sacrifice some query flexibility, but we can work around that with secondary indexes."

When to Use: Decision matrices are overkill for binary choices (SQL vs NoSQL). Use them when comparing 3+ options and the interviewer asks "How did you decide?" It shows structured thinking.

Anti-Patterns (What NOT to Do)

Anti-Pattern Why It's Bad Fix
"It depends" (and stopping) Shows indecisiveness, not helpful "It depends on [X]. Assuming [Y], I would choose [Z]."
Over-optimizing one dimension "Let's use Redis for everything!" Acknowledge limits: "Redis is fast but volatile. Need persistent storage too."
Ignoring constraints Designing multi-region for 1K users Match complexity to scale: "At 1K users, single region is fine."
Presenting one option "We'll use Kafka." (No alternatives) "We could use Kafka or RabbitMQ. Here's why I prefer Kafka..."
No mitigation for risks "Eventual consistency is a problem." (End) "To mitigate, we add version numbers and show 'syncing' indicator."

Practice Problems (Scenario-Based Trade-Offs)

Problem 1: Notification System

Scenario: You're building a notification system. Users can receive notifications via email, SMS, and push. Should you use a push or pull model?

Model Answer (STAR):

State: "We have two options. Push: server pushes notifications when events occur. Pull: clients poll for notifications periodically."

Trade-offs: "Push is real-time (instant delivery) but requires persistent connections (WebSocket or FCM for mobile). Pull is simpler (REST API) but delayed by polling interval (e.g., 30 sec)."

Argue: "For mobile push notifications, I'd use push (via FCM/APNs) because users expect instant alerts. For email, I'd use a queue-based system (async, doesn't need real-time)."

Risks: "Push notifications can fail if device is offline. Mitigation: store undelivered notifications in a queue with retry logic. When device reconnects, drain the queue."

Problem 2: Search Indexing

Scenario: You're indexing 10M documents for search. Should you index synchronously (on write) or asynchronously (background job)?

Model Answer:

State: "Synchronous: index immediately during write. Asynchronous: queue the write, index in background."

Trade-offs: "Synchronous guarantees search results are up-to-date, but slows down writes (indexing takes 100ms+). Asynchronous keeps writes fast (< 10ms), but search has a delay (eventual consistency)."

Argue: "For a document management system where users upload files, I'd use asynchronous. A 10-second delay before searchability is acceptable. For a chat app where users search their own messages, I'd use synchronous—users expect to search what they just sent."

Risks: "With async, if the worker crashes, documents aren't indexed. Mitigation: use a durable queue (SQS, Kafka) with retries and dead-letter queue for failed indexing."

Problem 3: Rate Limiting

Scenario: Should you implement rate limiting at the API gateway or at each service?

Model Answer:

State: "Centralized (API gateway) vs distributed (each service)."

Trade-offs: "Centralized is simpler (one place to manage) and protects the entire system. Distributed allows per-service policies (e.g., auth service can have stricter limits than search service) but requires coordination (shared state in Redis)."

Argue: "I'd start with centralized at the API gateway (using token bucket or sliding window). If we later need service-specific limits, add distributed rate limiting using Redis."

Risks: "If gateway becomes a bottleneck, we shard by user ID (route users to different gateway instances). Each instance tracks its own limits, and we accept slight overage (e.g., 105 requests instead of 100) for simplicity."

Problem 4: Data Retention

Scenario: Users upload 1 PB of data per year. Should you delete old data or archive it?

Model Answer:

State: "Delete: free up storage, lower cost. Archive: preserve data, enable compliance/analytics."

Trade-offs: "Deleting saves cost (1 PB/year × $23/TB/month = $23K/month). Archiving (S3 Glacier) costs $1/TB/month (23x cheaper) but retrieval is slow (hours)."

Argue: "For user-generated content (photos), I'd archive. Users rarely access old photos, but when they do, they tolerate delay. For logs, I'd delete after 90 days—logs older than 3 months are rarely useful."

Risks: "If compliance requires 7-year retention, we must archive. To reduce cost, compress before archiving (50% reduction) and use lifecycle policies (auto-move to Glacier after 30 days)."

Problem 5: Multi-Tenancy

Scenario: You're building a SaaS app. Should each customer get their own database or share one?

Model Answer:

State: "Shared database (multi-tenant) vs isolated database (single-tenant per customer)."

Trade-offs: "Shared is cost-efficient (1 database for 1000 customers) and easy to maintain (one schema upgrade). Isolated is secure (data breach doesn't affect others) and allows per-customer customization."

Argue: "For SMB customers, I'd use shared with tenant_id column (row-level isolation). For enterprise customers (who demand compliance), I'd offer isolated databases."

Risks: "Shared: noisy neighbor problem (one customer runs expensive query, slows others). Mitigation: connection pooling per tenant and query timeout. Isolated: high ops cost (1000 databases). Mitigation: automate provisioning with Terraform."

Summary

Final Advice: Trade-offs are where you prove seniority. Juniors give one answer. Mid-level gives pros/cons. Seniors give 2-3 options with context-driven justification. Staff engineers discuss second-order effects (e.g., "This choice is right today, but in 2 years when we're 10x scale, we'll need to migrate to Y. Here's the migration path..."). Show that level of thinking.