Chapter 62: Design Reddit

Community-driven content aggregation with voting, threaded discussions, and personalized feeds

1. Requirements

Functional

Feature	Description
Posts	Create text/link/media posts in subreddits
Comments	Threaded replies, unlimited depth, collapse/expand
Voting	Upvote/downvote posts and comments
Subreddits	Topic-based communities, moderation, rules
Feeds	Hot, New, Top (hour/day/week/month/all), Rising
Home Feed	Merged feed from subscribed subreddits

Non-Functional

Metric	Target
Scale	500M MAU, 100M DAU, 2M posts/day, 50M comments/day
Read:Write	100:1 (read-heavy)
Latency	Feed: <200ms, Vote: <50ms, Comment load: <300ms
Consistency	Eventual for votes/scores, strong for post/comment creation

Interview Tip: Clarify vote visibility delay tolerance (real-time vs. eventual), feed personalization depth (subscriptions only vs. ML recommendations), and moderation requirements (auto-mod, shadow bans).

2. High-Level Design

graph TB Client[Client App] subgraph "API Layer" API[API Gateway] PostSvc[Post Service] CommentSvc[Comment Service] VoteSvc[Vote Service] FeedSvc[Feed Service] UserSvc[User Service] end subgraph "Storage" PostDB[(Post DB
Cassandra)] CommentDB[(Comment DB
PostgreSQL)] VoteCache[(Vote Cache
Redis)] VoteDB[(Vote DB
Cassandra)] UserDB[(User DB
PostgreSQL)] FeedCache[(Feed Cache
Redis)] end subgraph "Processing" RankWorker[Ranking Worker] FeedWorker[Feed Worker] MsgQueue[Message Queue] end CDN[CDN] Client --> CDN Client --> API API --> PostSvc API --> CommentSvc API --> VoteSvc API --> FeedSvc API --> UserSvc PostSvc --> PostDB CommentSvc --> CommentDB VoteSvc --> VoteCache VoteSvc --> VoteDB UserSvc --> UserDB FeedSvc --> FeedCache VoteSvc --> MsgQueue MsgQueue --> RankWorker MsgQueue --> FeedWorker RankWorker --> PostDB FeedWorker --> FeedCache

Interview Tip: Emphasize separation of vote ingestion (low-latency writes) from ranking computation (async batch). Feed cache pre-computation avoids aggregating thousands of posts per request.

3. Voting System

Vote Storage

Layer	Technology	Purpose
Cache	Redis (vote counts)	Atomic INCR/DECR, fast reads
Log	Kafka (vote events)	Audit, anti-gaming analysis
DB	Cassandra (user_id, post_id, vote)	Persistent record, de-duplication

Vote Flow

1. User votes:
   - Check Redis: key = "user:123:post:456" (prevent double-vote)
   - If new vote: INCR "post:456:score", SET "user:123:post:456" = 1
   - If flip (up→down): DECR twice, update user vote
   - Async: Write to Cassandra, publish to Kafka

2. Score read:
   - GET "post:456:score" from Redis (cache hit ~99%)
   - Miss: Load from Cassandra, warm cache

3. Anti-gaming:
   - Rate limit: 100 votes/hour per user
   - Bot detection: ML model on vote patterns (Kafka stream)
   - Vote fuzzing: Display score ±5% to prevent manipulation feedback

Interview Tip: Redis atomic operations prevent race conditions on vote counts. Kafka stream enables real-time fraud detection without slowing vote API. Discuss trade-offs: strong consistency (slower) vs. eventual (faster, risk of double-vote during cache miss).

4. Ranking Algorithms

Hot Ranking (Posts)

Hot Score = log₁₀(max(|score|, 1)) × sign(score) + (time / 45000)

score = upvotes - downvotes
log₁₀ = logarithmic decay (first 10 votes = next 100 votes)
time = epoch seconds since 2005-12-08 (Reddit epoch)
45000s = 12.5 hours (time decay factor)

Result: New posts with modest votes can beat old posts with many votes.

Best Ranking (Comments - Wilson Score)

Wilson Lower Bound = (p̂ + z²/2n - z × sqrt[p̂(1-p̂)/n + z²/4n²]) / (1 + z²/n)

p̂ = upvotes / (upvotes + downvotes)
n = total votes
z = 1.96 (95% confidence)

Why: Balances confidence and proportion. 5 upvotes (100%) ranks below 90 upvotes / 100 total (90%).

Other Rankings

Sort	Formula
New	ORDER BY created_at DESC
Top	ORDER BY score DESC + time filter (hour/day/week/month/all)
Rising	Hot score for posts <8 hours old
Controversial	High vote count + score near zero (upvotes ≈ downvotes)

Interview Tip: Hot uses time and score (balanced), Top uses score only (all-time greats). Wilson score prevents "1 upvote, no downvotes = 100%" issue for comments.

5. Threaded Comments

Storage: Materialized Path

Column	Example	Purpose
comment_id	789	Primary key
post_id	456	Parent post
path	100.200.789	Ancestry chain (root → parent → self)
depth	2	Nesting level (0 = top-level)
score	42	Cached vote count
created_at	2026-05-12 10:30	Timestamp

Query Patterns

-- Load top-level comments (depth=0), sorted by Best
SELECT * FROM comments
WHERE post_id = 456 AND depth = 0
ORDER BY wilson_score DESC
LIMIT 20;

-- Load replies to comment 100 (depth=1)
SELECT * FROM comments
WHERE post_id = 456 AND path LIKE '100.%' AND depth = 1
ORDER BY wilson_score DESC;

-- Count total replies under comment 100
SELECT COUNT(*) FROM comments
WHERE post_id = 456 AND path LIKE '100.%';

Optimization: Collapse Deep Threads

Default load: Depth 0-2 (top + 2 levels)
Lazy load: "Continue this thread →" link for depth ≥ 3
Pagination: "Load more replies" (20 per page)
Index: (post_id, path, depth) for fast prefix scans

Interview Tip: Materialized path beats adjacency list (parent_id only) for tree queries. Alternative: Closure table (stores all ancestor-descendant pairs, faster reads, slower writes). Discuss trade-offs for 1000+ comment threads.

6. Feed Generation

Subreddit Feeds (Simple)

-- Hot feed for r/programming
SELECT * FROM posts
WHERE subreddit_id = 789
ORDER BY hot_score DESC
LIMIT 25;

-- Cached in Redis: "feed:subreddit:789:hot" → [post_id list]
-- TTL: 5 minutes (recompute via cron)

Home Feed (Complex)

Step	Process
1. Fetch Subs	User 123 → subscribed to [r/A, r/B, r/C]
2. Get Top Posts	From each sub's hot feed cache (Redis): top 100 posts
3. Merge + Sort	Combine 300 posts, sort by hot_score DESC
4. Filter	Remove hidden/blocked users, NSFW (if opt-out)
5. Paginate	Return top 25, cache result for 5 min

Pre-Computation

Per-sub feeds: Background job every 5 min (top 1000 posts by hot/new/top)
Home feed: On-demand (too many user combos to pre-compute)
Optimization: For users with 100+ subs, sample 50 most-active subs

Interview Tip: Pre-compute per-subreddit feeds (few subreddits, many users). Merge at request time for home feed (millions of unique subscription combos). Discuss ML personalization: beyond subscriptions, rank by user interests (click history, dwell time).

7. Database Design

Schema Overview

Table	DB	Key	Sharding
posts	Cassandra	(subreddit_id, created_at, post_id)	By subreddit_id (hot spots for large subs)
comments	PostgreSQL	post_id + path	By post_id hash
votes	Cassandra	(user_id, post_id)	By user_id (lookup: "did user vote?")
users	PostgreSQL	user_id	By user_id hash
subreddits	PostgreSQL	subreddit_id	Small, replicated
subscriptions	Cassandra	(user_id, subreddit_id)	By user_id

Sharding Strategy

Entity	Strategy	Reason
Posts	Consistent hash by subreddit_id	Co-locate posts in same sub (feed queries hit 1 shard)
Comments	Hash by post_id	All comments for a post in 1 shard (thread query hits 1 node)
Votes	Hash by user_id	Fast lookup: "did user vote on post X?" (user-centric query)
Users	Hash by user_id	Profile reads/writes isolated per user

Hot Shard Mitigation (Large Subreddits)

Problem: r/all, r/funny have 10M+ posts → single shard overload
Solution: Composite key (subreddit_id, time_bucket, post_id) where time_bucket = month
Feed query: Scatter-gather across last 3 months' buckets, merge results

Interview Tip: Cassandra for high-write (posts, votes), PostgreSQL for relational (comments tree, users). Discuss denormalization: store hot_score in posts table (avoid join with votes table). Cache invalidation: when vote count changes, async update post score.

8. Scalability

Scale Bottlenecks + Solutions

Bottleneck	Solution	Impact
Feed reads (100:1 read:write)	CDN for static assets, Redis for feed cache	99% cache hit → 100x load reduction
Vote count updates	Redis atomic ops + async DB write	Sub-10ms vote latency
Hot ranking computation	Pre-compute every 5 min (batch job)	Avoid per-request score calc for 1M posts
Comment tree queries	Materialized path + index on (post_id, path)	O(log N) vs. O(N) for adjacency list
Home feed (50+ subs)	Pre-fetch top 100 posts/sub from cache, merge in-memory	200ms vs. 5s for DB scatter-gather
Large subreddit sharding	Time-bucketed composite keys (month-level)	Distribute 10M posts across 12+ shards

Caching Layers

L1: CDN (static content, post thumbnails, avatars)
    - Cache-Control: max-age=86400 (1 day)

L2: Redis (hot data)
    - Vote counts: "post:456:score" → 1234 (TTL: none, write-through)
    - Feed cache: "feed:subreddit:789:hot" → [post IDs] (TTL: 5 min)
    - User session: "session:abc123" → user_id (TTL: 24 hours)

L3: Application cache (in-memory)
    - Subreddit metadata (small, replicated across all API servers)
    - User subscriptions (LRU cache, 10K entries)

Write Path Optimization

Votes: Write to Redis (sync), Cassandra (async via Kafka)
Posts: Write to Cassandra (sync), trigger feed cache invalidation (async)
Comments: Write to PostgreSQL (sync), no cache (always fresh)

Read Path Optimization

Feed: Redis → Cassandra (miss) → populate cache
Post detail: Cassandra (post) + Redis (vote count) + PostgreSQL (comments)
Home feed: Redis (per-sub feeds) → merge → filter → paginate

Interview Tip: Emphasize cache TTLs matched to data freshness needs (votes: real-time via Redis, feeds: 5 min stale OK). Discuss thundering herd: when popular post cache expires, 1000s of requests hit DB → solution: cache warming (refresh before expiry) or request coalescing.

Capacity Estimates

Metric	Calculation	Result
Post writes	2M/day = 23/sec	Trivial (1 Cassandra node handles 10K writes/sec)
Comment writes	50M/day = 578/sec	5-10 PostgreSQL shards
Vote writes	1B/day = 11.5K/sec	10 Redis nodes (1K writes/sec each)
Feed reads	100M DAU × 50 views/day = 5B/day = 57K/sec	50 Redis nodes (99% cache hit) + CDN
Storage	Posts: 5KB × 2M/day = 10GB/day = 3.6TB/year Comments: 500B × 50M/day = 25GB/day = 9TB/year	50TB total (5 years, with replication)

Summary

Voting: Redis atomic ops (low latency) + Cassandra persistence + Kafka anti-gaming
Ranking: Hot = log(score) + time decay, Wilson score for comments (confidence-weighted)
Comments: Materialized path storage (prefix scans), collapse depth ≥3, paginate replies
Feeds: Pre-compute per-subreddit (Redis cache), merge on-demand for home feed
Sharding: Posts by subreddit_id, comments by post_id, votes by user_id
Scale: 100:1 read:write → CDN + Redis cache (99% hit rate), async ranking, batch feed computation