Chapter 55: Design TikTok / Recommendation Engine
1. Requirements
| Category | Requirement | Target |
|---|---|---|
| Scale | Monthly Active Users | 1B MAU |
| Daily Video Uploads | 100M videos/day | |
| Feed Requests | 10B requests/day | |
| Performance | Recommendation Latency | <100ms (p99) |
| Video Load Time | <200ms | |
| Functional | For You Page | Personalized feed |
| Engagement Signals | Like, share, comment, watch%, skip | |
| Real-Time Adaptation | Immediate feedback loop | |
| Business | Watch Time | Maximize daily watch time |
| Retention | Next-day retention >40% |
Interview Callout: Clarify whether design focuses on serving layer (latency-critical) vs training pipeline (batch throughput). TikTok's uniqueness: extremely short videos mean users can consume 100+ videos/session, requiring rapid re-ranking.
2. High-Level Design
graph TB
subgraph Upload["Video Ingestion"]
A[Mobile App] -->|Upload| B[CDN/Storage]
B --> C[Transcode Pipeline]
C --> D[Content Moderation]
D --> E[Video Metadata Store]
end
subgraph Features["Feature Engineering"]
E --> F[Feature Store]
L[User Events] --> F
F --> G[(Redis
Real-Time Features)] F --> H[(Cassandra
Batch Features)] end subgraph Training["Model Training"] H --> I[Offline Training
Wide&Deep Model] I --> J[Model Registry] end subgraph Serving["Recommendation Serving"] K[API Gateway] --> M[Candidate Generation
ANN + CF] M --> N[Ranking Service] J -->|Load Model| N G --> N N --> O[Re-Ranking
Diversity/Freshness] O --> P[Feed Response] end L[User Events] -->|Real-Time| N P --> K
Real-Time Features)] F --> H[(Cassandra
Batch Features)] end subgraph Training["Model Training"] H --> I[Offline Training
Wide&Deep Model] I --> J[Model Registry] end subgraph Serving["Recommendation Serving"] K[API Gateway] --> M[Candidate Generation
ANN + CF] M --> N[Ranking Service] J -->|Load Model| N G --> N N --> O[Re-Ranking
Diversity/Freshness] O --> P[Feed Response] end L[User Events] -->|Real-Time| N P --> K
Interview Callout: Key bottleneck is candidate generation from billions of videos. Cannot score all videos - must reduce to 500-1000 candidates before ranking. Use multi-stage funnel: recall (fast, approximate) then ranking (slow, accurate).
3. Video Ingestion Pipeline
| Stage | Technology | Purpose | Details |
|---|---|---|---|
| Upload | S3 / Cloud Storage | Raw video storage | Multi-region, CDN distribution |
| Transcode | FFmpeg / AWS MediaConvert | Generate multiple bitrates | 480p, 720p, 1080p for ABR streaming |
| Thumbnail | Computer Vision | Extract key frames | 3-5 representative thumbnails |
| Moderation | AWS Rekognition + Manual | NSFW, violence, policy checks | Auto-reject + human review queue |
| Metadata Extraction | NLP + CV Models | Tags, objects, music, text | Scene detection, audio fingerprint, OCR |
| Indexing | Elasticsearch | Search by hashtag, audio, user | Inverted index + vector embeddings |
4. Feature Store
| Feature Type | Examples | Storage | Freshness |
|---|---|---|---|
| User Demographics | Age, location, language, device | Cassandra | Static/Daily |
| User Historical | Liked categories, avg watch%, session length | Cassandra | Hourly batch |
| User Real-Time | Last 10 watched videos, current session state | Redis | <1s |
| Video Metadata | Duration, upload time, hashtags, audio ID | Cassandra | Static |
| Video Statistics | Total views, like rate, completion rate | Cassandra | Hourly batch |
| Video Real-Time | Last hour engagement (velocity) | Redis | <5min |
| Contextual | Time of day, day of week, trending topics | Redis | <1min |
| Embeddings | Video embedding (768-dim), user embedding | Vector DB (Pinecone/Milvus) | Hourly |
Interview Callout: Feature store must support BOTH online serving (low-latency point lookups) and offline training (batch scan). Common pattern: write features to Kafka, dual-consume to Redis (online) and S3/Hive (offline).
5. Recommendation Pipeline
graph LR
A[Feed Request
user_id, context] --> B[Candidate Generation] B -->|500-1000 videos| C[Feature Retrieval] C --> D[Ranking Model] D -->|Top 50 scored| E[Re-Ranking] E -->|Final 20 videos| F[Response] B --> B1[CF: Similar Users] B --> B2[Content: Same Hashtag] B --> B3[ANN: Embedding Search] B --> B4[Trending: High Velocity] E --> E1[Diversity Filter] E --> E2[Freshness Boost] E --> E3[Creator Dedup]
user_id, context] --> B[Candidate Generation] B -->|500-1000 videos| C[Feature Retrieval] C --> D[Ranking Model] D -->|Top 50 scored| E[Re-Ranking] E -->|Final 20 videos| F[Response] B --> B1[CF: Similar Users] B --> B2[Content: Same Hashtag] B --> B3[ANN: Embedding Search] B --> B4[Trending: High Velocity] E --> E1[Diversity Filter] E --> E2[Freshness Boost] E --> E3[Creator Dedup]
| Stage | Input Size | Output Size | Latency Budget |
|---|---|---|---|
| Candidate Generation | 1B+ videos | 500-1000 | <30ms |
| Ranking | 500-1000 | 50-100 | <50ms |
| Re-Ranking | 50-100 | 20-30 | <10ms |
6. Candidate Generation
| Strategy | Method | Pros | Cons |
|---|---|---|---|
| Collaborative Filtering | User-User CF: Find similar users, get their liked videos | Personalized, leverages crowd wisdom | Cold start, popularity bias |
| Item-Item CF: "Users who liked X also liked Y" | Fast precomputation | Stale, filter bubble | |
| Content-Based | Same hashtags, audio, creator as past likes | Explainable, no cold start for videos | Limited serendipity |
| ANN (Embedding) | FAISS/HNSW: Find videos near user embedding | Semantic similarity, scales well | Requires good embeddings |
| Trending/Viral | High engagement velocity (last 1-6 hours) | Captures zeitgeist, viral content | Not personalized |
| Follower Feed | Videos from followed creators | User intent, high CTR | Small set for new users |
Interview Callout: TikTok blends multiple sources (each returns 100-200 candidates), then deduplicates. Key innovation: ANN with user/video embeddings learned from interactions. FAISS with HNSW index for <10ms lookup across billions of vectors.
7. Ranking Model
Model Architecture: Wide & Deep
| Component | Purpose | Features |
|---|---|---|
| Wide (Linear) | Memorization: Learn explicit rules | Cross features: (user_age=18-24) AND (video_category=dance) |
| Deep (DNN) | Generalization: Learn embeddings | User/video embeddings, dense features |
| Output | Multi-objective | P(watch), P(like), P(share), P(finish), P(comment) |
Key Features (Input to Model)
| Feature Group | Examples |
|---|---|
| User | Age, location, past 50 video embeddings, avg watch%, active hours |
| Video | Duration, upload recency, hashtags, audio ID, engagement rate, category |
| Context | Time of day, device, network speed, user session length so far |
| Cross | User embedding · video embedding (dot product), category match |
Training Pipeline
- Data Collection: User interactions streamed to Kafka (watch%, like, share, skip)
- Label Generation: Positive = watched >50% or liked. Negative = skipped <2s
- Training: Spark job processes 30 days of data daily, trains TensorFlow model (8 hours)
- Deployment: Model pushed to TorchServe/TF Serving, A/B test, gradual rollout
Interview Callout: Multi-objective optimization is critical - optimize for BOTH engagement (watch time) AND satisfaction (completion rate). Use weighted sum or Pareto-optimal models. Watch out for clickbait maximizing watch-start but low completion.
8. Real-Time Signals & Online Learning
| Signal | Latency | Action |
|---|---|---|
| Watch % > 80% | <1s | Show more from same creator, category, audio |
| Skip < 2s | <1s | Downrank similar videos in session |
| Replay | <1s | Strong positive signal - boost weight |
| Like/Share | <1s | Update user profile embedding immediately |
| Comment | <5s | Highest engagement - prioritize similar content |
Online Learning Approach
- Session-Based Updates: Adjust weights in-memory during user session (temporary)
- Micro-Batching: Every 5 minutes, retrain a lightweight model on latest interactions
- Bandit Algorithms: Explore/exploit for cold-start videos (Thompson Sampling)
- Feature Store Sync: Redis updated with last 10 interactions per user (<1s propagation)
Interview Callout: TikTok's competitive edge is real-time adaptation. Unlike YouTube (refresh every few hours), TikTok adjusts WITHIN the session. Use stateful serving: maintain session state in-memory, apply immediate re-ranking based on last 3-5 video reactions.
9. A/B Testing & Experimentation
| Component | Details |
|---|---|
| Framework | Feature flags (LaunchDarkly), user bucketing (hash user_id % 100) |
| Metrics (Primary) | Daily watch time, next-day retention, videos per session |
| Metrics (Secondary) | Like rate, share rate, completion rate, app uninstalls |
| Guardrail Metrics | P99 latency, error rate, video load failures |
| Test Duration | 7-14 days (account for weekly seasonality) |
| Sample Size | 1-5% of users per variant (1M+ users for statistical power) |
| Rollout Strategy | 1% → 5% → 25% → 50% → 100% over 2 weeks |
Interview Callout: Beware local optima - optimizing for immediate watch time can hurt long-term retention (addictive but low-quality content). Monitor retention cohorts, not just session metrics. Use holdout groups to measure long-term impact.
10. Cold Start Problem
New User Cold Start
| Stage | Strategy | Details |
|---|---|---|
| Onboarding (0-5 videos) | Demographic-based + Trending | Show popular videos for user's age/location, diverse categories |
| Exploration (5-20 videos) | Multi-armed bandit (Thompson Sampling) | Rapidly explore categories, track which get engagement |
| Exploitation (20+ videos) | Personalized model | Switch to full ranking model with learned preferences |
New Video Cold Start
| Strategy | Details |
|---|---|
| Initial Boost | Every new video shown to 100-500 random users (matched by basic metadata) |
| Performance Gating | If watch% > 50% in first 100 views → promote to wider audience |
| Contextual Bandit | Use video metadata (hashtags, audio) to select seed audience intelligently |
| Creator History | If creator has followers or past viral videos, boost initial distribution |
| Time Decay | Videos >48 hours old without traction → deprioritize |
Interview Callout: TikTok's "democratic" algorithm gives every video a chance (unlike follower-count-based platforms). Key: rapid experimentation - within minutes of upload, system knows if video is high-potential. Use contextual bandits, NOT pure random exploration (wastes user attention).
Key Takeaways
- Multi-Stage Funnel: Billion-scale corpus → 1K candidates → 50 ranked → 20 final (latency budget allocation)
- Real-Time Adaptation: Adjust recommendations WITHIN session based on immediate feedback (not just batch updates)
- Hybrid Candidate Generation: Blend CF + content-based + ANN + trending to maximize recall diversity
- Multi-Objective Ranking: Optimize for engagement AND satisfaction, avoid clickbait traps
- Cold Start = Exploration: Use bandits for new users/videos, balance explore/exploit
- Feature Store: Dual-path (online Redis + offline Cassandra/S3) to serve training + inference
- A/B Testing: Measure long-term retention, not just session metrics; beware local optima
Capacity Estimation
| Metric | Value | Calculation |
|---|---|---|
| Monthly Active Users | 1B MAU | Given |
| Daily Active Users | 700M DAU | 70% of MAU |
| Avg Watch Time | 95 min/day per user | Industry benchmark |
| Videos Uploaded | 5M videos/day | 0.7% of DAU upload |
| Videos per Session | 150 videos/session | 95 min / 38 sec avg video |
| Video Views | 105B views/day = 1.2M views/sec | 700M DAU × 150 videos |
| Storage (Daily) | 750TB/day | 5M videos × 3 resolutions × 50MB avg |
| Storage (Annual) | 274PB/year | 750TB × 365 |
| Recommendation QPS | 40K requests/sec | 700M sessions × 5 refreshes / 86400 sec |
| Interaction Events | 1.2M events/sec | Video views + likes + shares + comments |
| CDN Bandwidth | 50 Tbps peak | 1.2M views/sec × 5 Mbps avg bitrate |
Interview Callout: Key insight - video views are 1000x higher than recommendation requests because users watch 20-30 videos per feed refresh. Bottleneck is NOT model serving (40K QPS manageable), but video delivery (1.2M concurrent streams). This is why CDN with 95%+ cache hit rate is critical - origin servers only handle 60K requests/sec.
Database Design
| Table/Store | Schema | Technology | Sharding Strategy | Read/Write Pattern |
|---|---|---|---|---|
| Videos | video_id (PK), creator_id, url, duration, upload_time, status, hashtags | Cassandra | Shard by video_id (hash) | Write: 58 writes/sec (5M/day) Read: 1.2M reads/sec (hot videos cached) |
| Users | user_id (PK), username, profile_pic, bio, created_at, location, age | PostgreSQL (replicated) | Shard by user_id | Write: Low (profile updates rare) Read: 40K reads/sec (recommendation requests) |
| Interactions | user_id, video_id, watch_time, liked (bool), shared (bool), timestamp | Cassandra (time-series) | Composite key: (user_id, timestamp) | Write: 1.2M writes/sec (every view/like) Read: Batch only (training pipeline) |
| User Embeddings | user_id → 256-dim vector | Redis | Hash slot sharding | Write: Hourly batch update Read: 40K reads/sec (recommendation) |
| Video Embeddings | video_id → 768-dim vector | Milvus / Pinecone (ANN) | Vector index (HNSW) | Write: 58 writes/sec (new videos) Read: 40K ANN searches/sec |
| Social Graph | follower_id, followee_id, timestamp | Neo4j / PostgreSQL | Shard by follower_id | Write: Low (follow actions) Read: 10K reads/sec (follower feed) |
| Session State | session_id → {last_10_videos, engagement_signals} | Redis (TTL 1 hour) | Session ID hash | Write: 1.2M writes/sec (every interaction) Read: 40K reads/sec (in-session) |
Interview Callout: Key trade-off - Interactions table is write-heavy (1.2M writes/sec) so use Cassandra with no joins. But Users table needs ACID for money-related features (tips, ads), so use PostgreSQL with read replicas. Feature stores (embeddings) use specialized DBs: Redis for point lookups (low-latency), Milvus for ANN search (semantic similarity). Avoid joins across stores - denormalize data into feature store.
Scalability
Database Scaling
- Shard Interactions by user_id: Write distribution across 1000+ nodes. Each node handles 1200 writes/sec. Use consistent hashing for balanced load.
- Read Replicas for Users DB: 10 read replicas (4K reads/sec each) behind load balancer. Master handles writes only.
- Cache Hot Videos: Top 1% of videos (watched 80% of time) cached in CDN + Redis. 95% cache hit rate reduces DB reads from 1.2M to 60K/sec.
Feature Store Scaling
- Redis Cluster (Online): 50-node cluster with hash slot sharding. Each node 64GB RAM, stores 10M user embeddings. Auto-failover with Sentinel.
- HDFS/S3 (Offline Batch): Store 90 days of interactions (10PB) for training. Spark reads in parallel from 1000 partitions.
- Lambda Architecture: Real-time path (Kafka → Flink → Redis) for <1s features. Batch path (HDFS → Spark → Cassandra) for hourly aggregates.
Model Serving Scaling
- GPU Cluster: 500 GPU nodes (NVIDIA A100) for ranking model inference. Each GPU handles 100 QPS, total 50K QPS capacity (40K + headroom).
- Auto-Scaling: Kubernetes HPA scales pods based on GPU utilization (>70% triggers scale-up). Pre-warm pools for faster spin-up.
- A/B Traffic Splitting: Envoy/Istio routes 5% to canary model, 95% to prod. Gradual rollout based on metrics.
- Model Versioning: Keep last 3 model versions deployed. Instant rollback if error rate spikes.
Video Delivery Scaling
- CDN Offload: CloudFront/Akamai handles 95% of 1.2M concurrent streams. Origin servers (S3) only serve cache misses (60K/sec).
- Adaptive Bitrate: HLS/DASH with 3 resolutions (480p/720p/1080p). Client selects based on network speed. Reduces bandwidth 30% vs fixed bitrate.
- Edge Caching: Videos cached at 200+ edge locations. 80% of users served from edge within 50ms RTT.
Event Streaming Scaling
- Kafka: 100-node cluster, 1000 partitions (interactions topic). Handles 1.2M events/sec write throughput. 7-day retention for replay.
- Flink (Real-Time): Consumes Kafka, computes sliding window aggregates (last 10 videos per user), outputs to Redis. Processes 1.2M events/sec with <500ms latency.
- Spark (Batch): Hourly jobs aggregate 4.3B interactions/hour into user/video stats. 500-node cluster completes in 15 minutes.
Multi-Region Strategy
- Training: Centralized in one region (US-East) with most compute. Train on global data, push models to all regions.
- Serving: Deploy models in 10 regions. Each region serves local users (latency <100ms). Async replication of user profiles.
- Content Storage: Videos replicated to 3 regions per geography (US, EU, APAC). CloudFront pulls from nearest origin.
- Feature Store: Redis replicated cross-region (eventual consistency OK). Cassandra multi-datacenter with local quorum writes.
Interview Callout: TikTok's scale challenge is NOT model complexity (Wide&Deep is standard), but THROUGHPUT - 1.2M interaction writes/sec and 40K recommendation inferences/sec. Key: offload video delivery to CDN (95% hit rate), shard interactions DB by user_id (1000+ nodes), use GPU cluster for model serving (500 GPUs), and Lambda architecture for feature store (real-time + batch paths). Multi-region for latency, but train centrally to avoid data fragmentation.
Trade-offs
| Decision | Option A | Option B | TikTok's Choice | Reasoning |
|---|---|---|---|---|
| Recommendation Freshness | Batch (hourly updates) Pros: Lower cost, simpler Cons: Stale recommendations |
Real-time (sub-second) Pros: Immediate personalization Cons: High compute cost, complex |
Hybrid (Lambda) | Real-time for session state (last 10 videos), batch for long-term stats. Balances freshness and cost. |
| Candidate Retrieval | Collaborative Filtering Pros: Leverages crowd wisdom Cons: Cold start, popularity bias |
Content-Based (embeddings) Pros: No cold start for videos Cons: Filter bubble, less serendipity |
Hybrid (both + trending) | Blend 4 sources (CF, ANN, trending, followers) - 200 candidates each. Maximizes recall diversity. |
| Ranking Model | Single Deep Model Pros: Lower latency (1 inference) Cons: Less accurate |
Ensemble (3 models) Pros: Higher accuracy (+3% engagement) Cons: 3x latency, complex deployment |
Single Wide&Deep | Latency critical (<50ms budget). Wide&Deep balances memorization + generalization in one model. |
| Ranking Scores | Pre-compute (offline) Pros: Fast serving (lookup) Cons: Stale (no real-time context) |
On-the-fly (online) Pros: Fresh, contextual Cons: High compute (40K inferences/sec) |
On-the-fly with caching | Compute scores online (context matters), but cache candidate lists (1-hour TTL). Freshness + speed. |
| Interactions Storage | SQL (PostgreSQL) Pros: ACID, joins, analytics Cons: Write bottleneck (1.2M/sec) |
NoSQL (Cassandra) Pros: Write throughput, partition-tolerant Cons: No joins, eventual consistency |
NoSQL (Cassandra) | 1.2M writes/sec requires horizontal write scaling. Interactions don't need joins - batch jobs aggregate. |
| Cold Start Strategy | Random Exploration Pros: Simple, unbiased Cons: Wastes user attention, slow learning |
Contextual Bandit Pros: Faster convergence, personalized Cons: Complex, needs good priors |
Contextual Bandit (Thompson Sampling) | Use metadata (age, location, initial hashtag clicks) to intelligently explore. Learn user preferences in 10-20 videos. |
Interview Callout - Interest Graph vs Social Graph: TikTok's competitive advantage is the "interest graph" (content-based recommendations) over Instagram/Facebook's "social graph" (follower-based feeds). Why? New user engagement: On Instagram, new users with 0 followers see empty feed (cold start problem). On TikTok, new users immediately see personalized content based on demographics + rapid exploration (10 videos to learn preferences). Content democratization: On Instagram, reach is gated by follower count. On TikTok, every video gets initial distribution (100-500 views) - if engagement is high, it goes viral regardless of creator size. Addiction loop: Social graph is bounded (finite friends), but interest graph is infinite (always new content). This drives TikTok's 95 min/day watch time vs Instagram's 30 min/day.