System Design Bible

Chapter 64: Design LinkedIn

1. Requirements

Functional Requirements

Non-Functional Requirements

Scale Estimation

Daily Active Users: 300M DAU
Connections per user: Average 500 connections
Feed posts: 100M posts/day
Messages: 500M messages/day
Searches: 200M searches/day
Storage: ~50 PB (profiles, posts, media, messages)

Capacity Estimation

Traffic

Operation Calculation QPS Peak QPS
Feed reads 300M × 10 views/day = 3B/day 35K QPS 70K QPS
PYMK queries 300M × 5/day = 1.5B/day 17K graph traversals/sec 34K graph traversals/sec
Messages (writes) 500M/day 6K writes/sec 12K writes/sec
Messages (reads) 10:1 read/write ratio 60K reads/sec 120K reads/sec

Storage

Data Type Calculation Total Size
Graph storage 900M users × 500 connections = 450B edges × 50 bytes 22TB
Feed cache 300M users × 1KB feed cache 300GB (fits in Redis cluster)
Search index 900M profiles + 58M companies + 20M jobs ~5TB Elasticsearch
Interview Callout: The graph traversal QPS (17K/sec) is the hardest to scale — each PYMK requires 2-hop BFS over 500+ connections. This is why LinkedIn invests heavily in specialized graph databases and caching strategies for connection recommendations.

2. High-Level Design

graph TB Client[Client Apps
Web/Mobile] subgraph API Gateway Layer Gateway[API Gateway
Rate Limiting] end subgraph Application Services ProfileSvc[Profile Service] ConnectionSvc[Connection Service] FeedSvc[Feed Service] MessageSvc[Message Service] JobSvc[Job Service] SearchSvc[Search Service] end subgraph Data Layer ProfileDB[(Profile DB
MySQL)] GraphDB[(Connection Graph
Neo4j/JanusGraph)] FeedCache[(Feed Cache
Redis)] MessageDB[(Message DB
Cassandra)] JobDB[(Job DB
PostgreSQL)] SearchIdx[(Search Index
Elasticsearch)] end subgraph Storage S3[Object Storage
S3/CDN
Profile Photos/Media] end subgraph Message Queue Kafka[Kafka
Event Streaming] end Client --> Gateway Gateway --> ProfileSvc Gateway --> ConnectionSvc Gateway --> FeedSvc Gateway --> MessageSvc Gateway --> JobSvc Gateway --> SearchSvc ProfileSvc --> ProfileDB ProfileSvc --> S3 ProfileSvc --> Kafka ConnectionSvc --> GraphDB ConnectionSvc --> Kafka FeedSvc --> FeedCache FeedSvc --> Kafka MessageSvc --> MessageDB MessageSvc --> Kafka JobSvc --> JobDB JobSvc --> SearchIdx SearchSvc --> SearchIdx SearchSvc --> GraphDB Kafka --> FeedSvc Kafka --> SearchIdx
Interview Tip: Start with core entities (User, Connection, Post, Message, Job) and their relationships. Clarify read/write patterns early - LinkedIn is heavily read-optimized with extensive caching.

3. Connection Graph

Graph Database Design

LinkedIn's connection network is fundamentally a graph problem. Use a specialized graph database like Neo4j or JanusGraph to efficiently query relationships.

Node and Edge Schema

// Node: User
{
  userId: "u123",
  name: "John Doe",
  headline: "Software Engineer",
  location: "San Francisco"
}

// Edge: Connection
{
  from: "u123",
  to: "u456",
  timestamp: 1714512000,
  status: "ACCEPTED"  // PENDING, ACCEPTED, BLOCKED
}

Degree-Based Querying

// Cypher query for 2nd degree connections
MATCH (me:User {id: $userId})-[:CONNECTED]->(:User)-[:CONNECTED]->(friend2)
WHERE NOT (me)-[:CONNECTED]->(friend2) AND me.id <> friend2.id
RETURN DISTINCT friend2
LIMIT 100

People You May Know (PYMK)

PYMK is a key growth feature using multiple signals:

graph LR You((You)) A((Alice)) B((Bob)) C((Carol)) D((Dave)) You ---|30 mutual| A You ---|20 mutual| B You ---|5 mutual| C A ---|works at| Same_Company[Same Company] You ---|works at| Same_Company B ---|studied at| Same_School[Same School] You ---|studied at| Same_School style A fill:#90EE90 style B fill:#87CEEB style C fill:#FFB6C1

Graph Partitioning

With 900M users and billions of connections, partition the graph:

Interview Tip: Explain why traditional RDBMS struggle with graph queries (multiple self-joins on Users table). Graph DBs use index-free adjacency for O(1) edge traversal.

4. Feed System

Feed Generation Approaches

Approach Pros Cons
Pull Model (Fan-out on read) Simple, no storage overhead Slow for users with many connections
Push Model (Fan-out on write) Fast reads (pre-computed) Expensive writes for influencers
Hybrid Model Best of both More complexity

LinkedIn's Hybrid Approach

Feed Ranking Algorithm

LinkedIn uses engagement-based ranking, not pure chronological:

FeedScore = W1 * Recency
          + W2 * ConnectionStrength
          + W3 * ContentRelevance
          + W4 * EngagementPrediction

// Factors:
// - Recency: Decay function (exponential or linear)
// - ConnectionStrength: 1st degree > 2nd degree, interaction history
// - ContentRelevance: Skills/interests match, hashtag similarity
// - EngagementPrediction: ML model predicting like/comment/share probability

Feed Architecture

sequenceDiagram participant User participant FeedService participant FeedCache participant RankingML participant PostDB User->>FeedService: GET /feed FeedService->>FeedCache: Get cached feed (top 1000) FeedCache-->>FeedService: [post_ids with scores] FeedService->>PostDB: Fetch post details (batch) PostDB-->>FeedService: Post content, metadata FeedService->>RankingML: Re-rank with real-time signals RankingML-->>FeedService: Ranked feed FeedService-->>User: Personalized feed (paginated)

Viral Content Detection

Detect trending posts and amplify them:

Interview Tip: Discuss trade-offs between chronological (simple, predictable) vs algorithmic feeds (higher engagement but filter bubble concerns). LinkedIn prioritizes professional relevance.

6. Messaging System

Real-Time Messaging Requirements

Architecture

graph TB User1[User 1] User2[User 2] subgraph WebSocket Layer WS1[WebSocket Server 1] WS2[WebSocket Server 2] end subgraph Message Service MsgRouter[Message Router] PresenceSvc[Presence Service
Redis] end subgraph Storage MessageDB[(Cassandra
Messages)] Redis[(Redis
Online Users)] end User1 <-->|WebSocket| WS1 User2 <-->|WebSocket| WS2 WS1 --> MsgRouter WS2 --> MsgRouter MsgRouter --> MessageDB MsgRouter --> PresenceSvc PresenceSvc --> Redis MsgRouter -.->|Route to WS2| WS2

Message Flow

  1. User1 sends message via WebSocket to WS1
  2. WS1 forwards to Message Router
  3. Router checks Presence Service - is User2 online?
  4. If online: look up User2's WebSocket server (WS2) in Redis, push message
  5. If offline: store in push notification queue
  6. Persist message to Cassandra (partition key: conversationId, sort key: timestamp)
  7. Send delivery receipt back to User1

Cassandra Schema

CREATE TABLE messages (
    conversation_id UUID,
    message_id TIMEUUID,
    sender_id UUID,
    content TEXT,
    created_at TIMESTAMP,
    read_by SET,
    PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

Read Receipts

Interview Tip: Discuss WebSocket vs long-polling trade-offs. WebSockets provide true bidirectional communication but require sticky sessions. Use a message broker (Redis Pub/Sub or Kafka) to route between WebSocket servers.

7. Job Matching and Recommendations

Job Recommendation System

LinkedIn's "Jobs You May Be Interested In" uses collaborative filtering and skills matching.

Candidate-Job Matching Score

MatchScore = W1 * SkillMatch
           + W2 * LocationFit
           + W3 * ExperienceFit
           + W4 * SalaryFit
           + W5 * CompanyFit

// SkillMatch: Jaccard similarity between candidate skills and job requirements
// LocationFit: Distance from job location (or remote flag)
// ExperienceFit: Years of experience vs required range
// SalaryFit: Candidate's expected salary vs job salary
// CompanyFit: Interest in company size, industry

Skills Graph

Build a skills ontology to understand skill relationships:

graph TD SWE[Software Engineering] Backend[Backend Development] Frontend[Frontend Development] ML[Machine Learning] Java[Java] Python[Python] Go[Go] React[React] Vue[Vue.js] Angular[Angular] DL[Deep Learning] NLP[Natural Language Processing] SWE --> Backend SWE --> Frontend SWE --> ML Backend --> Java Backend --> Python Backend --> Go Frontend --> React Frontend --> Vue Frontend --> Angular ML --> DL ML --> NLP ML --> Python

Application Tracking

Recruiter Tools

8. Scalability and Performance

Graph Database Sharding

Feed Cache Strategy

Search Cluster Scaling

CDN and Media Storage

Database Choices

Data Type Database Reason
Profiles MySQL/PostgreSQL Structured, ACID transactions
Connections Neo4j/JanusGraph Graph queries (PYMK, degrees)
Messages Cassandra High write throughput, time-series
Feed Cache Redis Sub-ms latency, sorted sets
Jobs PostgreSQL Complex queries, ACID
Search Elasticsearch Full-text search, ranking

Rate Limiting

Interview Tip: When discussing scalability, always tie it back to the requirements (900M users, read-heavy). Mention monitoring and observability - use Prometheus/Grafana to track feed latency, cache hit rates, search QPS.

Summary

Designing LinkedIn requires handling multiple complex subsystems:

Key scalability patterns: aggressive caching (Redis for feeds), specialized databases (graph DB for connections, Cassandra for messages), and horizontal sharding across all layers.