Chapter 56: Design Zoom/Video Conferencing System

1. Requirements

Functional Requirements

Feature	Specification	Priority
1:1 Video Calls	Peer-to-peer video/audio communication	P0
Group Video Calls	Up to 1000 participants per meeting	P0
Screen Sharing	Share screen/application window, high resolution	P0
Meeting Recording	Server-side recording, cloud storage, playback	P1
Chat	Text messages during meeting, file sharing	P1
Active Speaker Detection	Highlight current speaker, optimize bandwidth	P1
Gallery View	Show multiple participants simultaneously	P1
Virtual Backgrounds	Background blur/replacement (client-side)	P2

Non-Functional Requirements

Requirement	Target	Rationale
Low Latency	<150ms end-to-end	Real-time conversation quality
High Availability	99.9% uptime	Mission-critical business tool
Scalability	300M daily participants	Global enterprise scale
Video Quality	720p/1080p HD, 30fps	Professional quality
Audio Quality	48kHz, opus codec	Clear communication
Network Adaptation	Graceful degradation 500kbps-4Mbps	Variable network conditions
Security	End-to-end encryption option, AES-256	Privacy/compliance

Scale Parameters

Metric	Value
Daily Active Meeting Participants	300 million
Max Participants per Meeting	1000 (video up to 49 visible)
Average Meeting Duration	45 minutes
Peak Concurrent Meetings	~10 million
Average Participants per Meeting	5

2. Capacity Estimation

Bandwidth Per User

Stream Type	Quality	Bitrate (Upload)	Bitrate (Download)
Video - Low	360p @ 15fps	500 kbps	-
Video - Medium	720p @ 30fps	1.5 Mbps	-
Video - High	1080p @ 30fps	3-4 Mbps	-
Audio	Opus 48kHz	50-100 kbps	50-100 kbps per speaker
Screen Share	1080p @ 5-10fps	1-2 Mbps	-

Concurrent Connection Capacity

Component	Calculation	Result
Daily Participants	300M given	300M
Avg Meeting Duration	45 minutes	0.75 hours
Peak Hour Concentration	~15% of daily in peak hour	45M participants
Peak Concurrent Users	45M * 0.75 / 1 hour	~34M concurrent
Avg Participants per Meeting	Assumed	5
Peak Concurrent Meetings	34M / 5	~7M meetings

Bandwidth Requirements

Scenario	Calculation	Total Bandwidth
User Upload (720p)	1.5 Mbps video + 100 kbps audio	~1.6 Mbps per user
User Download (5-person meeting)	4 streams @ 1.5 Mbps + 4 audio @ 100 kbps	~6.4 Mbps per user
Gallery View (25 visible)	25 streams @ 360p (300 kbps each)	~7.5 Mbps download
Peak Infrastructure Bandwidth	34M users * 1.6 Mbps upload	~54 Tbps ingress
Peak Infrastructure Bandwidth	34M users * 6.4 Mbps download	~218 Tbps egress

Storage (Recording)

Metric	Calculation	Result
Recording Rate	~5% of meetings recorded	350K concurrent recordings
Recording Bitrate	2 Mbps (compressed composition)	2 Mbps per recording
Daily Recording Volume	350K * 45 min * 2 Mbps	~4.7 PB/day
90-day Retention	4.7 PB * 90	~423 PB storage

3. High-Level Design

System Architecture

graph TB subgraph "Client Layer" C1[Web Client
WebRTC] C2[Desktop Client
WebRTC] C3[Mobile Client
WebRTC] end subgraph "Edge Layer - Regional PoPs" STUN[STUN Server
NAT Discovery] TURN[TURN Server
Relay Fallback] SIG[Signaling Server
WebSocket/SIP] end subgraph "Media Layer" SFU1[SFU - Region 1
Media Routing] SFU2[SFU - Region 2
Media Routing] SFU3[SFU - Region 3
Media Routing] CASCADE[Cascading SFU
Inter-region] end subgraph "Application Layer" API[API Gateway] MEETING[Meeting Service
Room Management] AUTH[Auth Service
JWT/OAuth] PRESENCE[Presence Service
User Status] end subgraph "Data Layer" REDIS[(Redis
Room State)] POSTGRES[(PostgreSQL
User/Meeting Data)] KAFKA[Kafka
Events] end subgraph "Recording & Storage" REC[Recording Service
Composition] TRANSCODE[Transcoding Pipeline
Multiple Formats] S3[(S3/Object Storage
Recordings)] end subgraph "Chat & Messaging" CHAT[Chat Service
WebSocket] CHATDB[(Cassandra
Chat History)] end C1 & C2 & C3 --> STUN C1 & C2 & C3 --> TURN C1 & C2 & C3 --> SIG C1 & C2 & C3 -.media.-> SFU1 C1 & C2 & C3 -.media.-> SFU2 SIG --> API API --> MEETING API --> AUTH API --> PRESENCE MEETING --> REDIS MEETING --> POSTGRES MEETING --> KAFKA SFU1 & SFU2 & SFU3 --> CASCADE SFU1 --> REC REC --> TRANSCODE TRANSCODE --> S3 C1 & C2 & C3 --> CHAT CHAT --> CHATDB CHAT --> KAFKA style SFU1 fill:#90EE90 style SFU2 fill:#90EE90 style SFU3 fill:#90EE90 style CASCADE fill:#FFD700

Component Responsibilities

Component	Responsibility	Technology
Signaling Server	SDP negotiation, ICE candidates exchange, room join/leave	WebSocket, Node.js/Go
STUN Server	Discover public IP/port for NAT traversal	coturn, restund
TURN Server	Relay media when P2P fails (symmetric NAT, firewall)	coturn, Pion TURN
SFU (Selective Forwarding Unit)	Receive media streams, selectively forward to participants	Janus, mediasoup, Pion
Meeting Service	Create/manage rooms, participant list, permissions	Microservice, gRPC
Recording Service	Composite video/audio, encode, upload to storage	FFmpeg, custom pipeline
Chat Service	Text messaging, file sharing, message history	WebSocket, Cassandra
Presence Service	Track online/offline/in-meeting status	Redis pub/sub

Data Flow

Flow Type	Path	Protocol
Signaling	Client ↔ Signaling Server ↔ Meeting Service	WebSocket, HTTPS
Media (P2P)	Client ↔ Client (1:1 only)	WebRTC (SRTP/UDP)
Media (Group)	Client ↔ SFU ↔ Clients	WebRTC (SRTP/UDP)
Media (Relay)	Client → TURN → SFU	TURN (UDP/TCP/TLS)
Chat	Client ↔ Chat Service	WebSocket
Recording	SFU → Recording Service → S3	RTP → FFmpeg → S3 API

4. WebRTC Foundation

WebRTC Overview

Aspect	Description	Key Points
What is WebRTC?	Open standard for real-time peer-to-peer communication	Built into browsers, no plugins needed
Media Transport	SRTP (Secure RTP) over UDP	Low latency, encrypted media
NAT Traversal	ICE (Interactive Connectivity Establishment)	Discovers best path through firewalls/NAT
Session Description	SDP (Session Description Protocol)	Negotiates codecs, resolutions, candidates
Signaling	Not specified by WebRTC standard	Typically WebSocket or SIP
Encryption	Mandatory DTLS-SRTP	All media encrypted by default

ICE/STUN/TURN - NAT Traversal

Protocol	Purpose	When Used	Cost
ICE	Framework to discover connection candidates	Always (orchestrates STUN/TURN)	Low
STUN	Discover public IP:port from behind NAT	Most connections (~80%)	Very Low
TURN	Relay media when direct connection fails	Symmetric NAT, restrictive firewall (~20%)	High (2x bandwidth)

SDP Offer/Answer Flow

sequenceDiagram participant A as Client A participant S as Signaling Server participant B as Client B Note over A: User initiates call A->>A: Create PeerConnection A->>A: Add local media tracks A->>A: createOffer() A->>A: setLocalDescription(offer) Note over A: Gather ICE candidates A->>S: Send SDP Offer + ICE candidates S->>B: Forward SDP Offer Note over B: Incoming call B->>B: Create PeerConnection B->>B: setRemoteDescription(offer) B->>B: Add local media tracks B->>B: createAnswer() B->>B: setLocalDescription(answer) B->>S: Send SDP Answer + ICE candidates S->>A: Forward SDP Answer A->>A: setRemoteDescription(answer) Note over A,B: ICE candidate exchange continues A->>S: Additional ICE candidates S->>B: Forward candidates B->>S: Additional ICE candidates S->>A: Forward candidates Note over A,B: ICE completes, media flows A-->>B: SRTP media (direct or via TURN) B-->>A: SRTP media (direct or via TURN)

Connection Establishment Steps

Step	Action	Details
1. Create Offer	Caller creates SDP offer	Lists supported codecs (VP8, H.264), resolutions, media types
2. Set Local Description	Caller sets offer as local SDP	Triggers ICE candidate gathering
3. Send Offer	Caller sends offer via signaling	WebSocket to signaling server to callee
4. Set Remote Description	Callee receives offer, sets as remote SDP	Understands caller's capabilities
5. Create Answer	Callee creates SDP answer	Agrees on codec/resolution intersection
6. Set Local Description	Callee sets answer as local SDP	Triggers ICE candidate gathering
7. Send Answer	Callee sends answer via signaling	Back to caller
8. ICE Candidate Exchange	Both sides send ICE candidates	host, srflx (STUN), relay (TURN) candidates
9. ICE Connectivity Checks	Try all candidate pairs (STUN checks)	Measures latency, prefers direct path
10. Media Flow	Best candidate pair selected, media starts	DTLS handshake → SRTP keys → encrypted media

5. Media Architecture: SFU vs MCU

Architecture Comparison

Aspect	SFU (Selective Forwarding Unit)	MCU (Multipoint Control Unit)
How it Works	Receives streams, forwards selectively without decoding	Decodes all streams, mixes/composes, encodes single stream
Server CPU	Low (no decode/encode)	Very High (decode N, encode 1)
Client Bandwidth (N participants)	High: receives N-1 streams	Low: receives 1 composed stream
Client CPU	Higher (decode multiple streams)	Lower (decode 1 stream)
Scalability	Excellent (stateless forwarding)	Poor (expensive per-room processing)
Flexibility	High (client controls layout)	Limited (server dictates layout)
Quality	High (original quality per stream)	Lower (re-encoding artifacts)
Use Case	Modern video conferencing (Zoom, Teams, Meet)	Legacy systems, recording, broadcasting

SFU Architecture (Preferred)

MCU Architecture (Legacy)

Why SFU is Preferred for Zoom

Reason	Benefit
No Transcoding	Lower server costs, higher quality (no re-encoding loss)
Stateless Forwarding	Easy horizontal scaling, simple load balancing
Client-Side Layout Control	Users customize view (speaker view, gallery view, pinning)
Simulcast Support	SFU can select quality layer per receiver (bandwidth adaptation)
Lower E2E Latency	No decode/encode delay at server
Modern Clients	Today's devices handle multi-stream decode efficiently

6. Quality Adaptation

Adaptive Bitrate Techniques

Technique	How It Works	Use Case
Simulcast	Sender encodes video at multiple qualities (e.g., 360p, 720p, 1080p) simultaneously, uploads all to SFU	SFU selects appropriate layer per receiver based on bandwidth/CPU
SVC (Scalable Video Coding)	Single encoded stream with multiple quality layers (base + enhancement layers)	More efficient than simulcast (less upload bandwidth), but lower browser support
Bandwidth Estimation	Monitor packet loss, RTT, receiver reports to estimate available bandwidth	Dynamically adjust sending bitrate
Congestion Control (GCC)	Google Congestion Control algorithm adjusts bitrate based on network feedback	Prevent overwhelming network, reduce packet loss
FEC (Forward Error Correction)	Send redundant packets to recover from loss without retransmission	Improves quality on lossy networks (cost: extra bandwidth)
Temporal/Spatial Layers	Drop frames (temporal) or reduce resolution (spatial) under congestion	Graceful degradation

Simulcast Architecture

Quality Adaptation Parameters

Parameter	Poor Network	Good Network	Excellent Network
Available Bandwidth	< 500 kbps	500 kbps - 2 Mbps	> 2 Mbps
Video Resolution	240p/360p	720p	1080p
Frame Rate	15 fps	30 fps	30 fps
Video Bitrate	300-500 kbps	1-1.5 Mbps	2-4 Mbps
Audio Codec	Opus 32 kbps	Opus 64 kbps	Opus 96-128 kbps
FEC	Enabled (5-10%)	Enabled (2-5%)	Disabled

Bandwidth Estimation (GCC)

Signal	Measurement	Action
Packet Loss	> 5% loss rate	Reduce bitrate 15-20%
RTT Increase	RTT grows > 50ms above baseline	Network congestion detected, reduce bitrate
Receiver Reports (RTCP)	Jitter, loss, delay stats	Adjust sending rate based on receiver feedback
Sending Rate Success	Low loss, stable RTT for 30s+	Gradually increase bitrate 5% every few seconds
Transport-Wide CC	Per-packet arrival times at receiver	Fine-grained congestion signal, adjust proactively

7. Scaling Group Calls

SFU Scaling for Large Meetings

Meeting Size	Architecture	Details
1:1 Call	Peer-to-peer (P2P)	Direct WebRTC, no server media routing, lowest latency
2-10 Participants	Single SFU	Each uploads 1 stream, downloads 1-9 streams, manageable bandwidth
10-50 Participants	Single SFU + Simulcast	SFU selects quality layer per receiver, gallery view shows subset
50-1000 Participants	Cascading SFUs	Regional SFUs connect to master SFU, reduce cross-region bandwidth
Large Webinars (>1000)	SFU + CDN (receive-only)	Presenters to SFU, attendees stream from CDN (HLS/DASH), 10-30s latency acceptable

Active Speaker Detection

Technique	How It Works	Benefit
Audio Level Analysis	Monitor RTP audio level extensions, detect loudest stream	SFU forwards active speaker at higher quality
Client-Side Detection	Client analyzes audio, highlights speaker in UI	Better UX, no server round-trip
Dominant Speaker Switching	SFU sends high-quality stream of active speaker, lower quality for others	Optimizes bandwidth usage
Smooth Transitions	Delay switching 1-2s to avoid rapid layout changes	Prevents jarring UI flips

Cascading SFU Architecture

graph TB subgraph "US-East Region" US1[Client US-1] US2[Client US-2] US3[Client US-3] SFU_US[SFU US-East] US1 & US2 & US3 <--> SFU_US end subgraph "EU-West Region" EU1[Client EU-1] EU2[Client EU-2] SFU_EU[SFU EU-West] EU1 & EU2 <--> SFU_EU end subgraph "Asia-Pacific Region" APAC1[Client APAC-1] APAC2[Client APAC-2] SFU_APAC[SFU APAC] APAC1 & APAC2 <--> SFU_APAC end MASTER[Master SFU
Room Coordinator] SFU_US <-->|Aggregate Streams| MASTER SFU_EU <-->|Aggregate Streams| MASTER SFU_APAC <-->|Aggregate Streams| MASTER style MASTER fill:#FFD700 style SFU_US fill:#90EE90 style SFU_EU fill:#90EE90 style SFU_APAC fill:#90EE90

Gallery View Optimization

Visible Participants	Strategy	Bandwidth Savings
1 (Speaker View)	Receive only active speaker at 1080p	~95% vs full gallery
4 (2x2 Grid)	Receive 4 streams at 720p	~80% vs all participants
9 (3x3 Grid)	Receive 9 streams at 480p	~60% vs all participants
25 (5x5 Grid)	Receive 25 streams at 360p	~20% vs all participants
49 (7x7 Grid)	Receive 49 streams at 180p (thumbnail)	Baseline (max feasible)
> 49 Participants	Show only 49, paginate, or show active speakers	Cap bandwidth regardless of total participants

Bandwidth Optimization Strategies

Strategy	Implementation	Impact
Last-N	Receive only last N active speakers	Reduces download to N streams regardless of room size
Pagination	Show participants page-by-page (e.g., 25 at a time)	User controls visible set
Downgrade Off-Screen	Request 180p for thumbnails, 720p for focused tile	Client-driven quality selection
Pause Invisible Streams	Stop receiving streams not in viewport	Aggressive bandwidth savings (UX tradeoff on scroll)
Audio-Only Mode	Disable video receive when user is bandwidth-constrained	Falls back to reliable audio experience

8. Recording

Recording Architecture

Component	Responsibility	Technology
SFU Tap	SFU forwards media streams to recording service	Duplicate RTP packets to recorder
Composition Engine	Decode all video/audio, composite into single video (grid/speaker layout)	FFmpeg, GStreamer
Encoder	Encode composed video to H.264/VP9	Hardware encoders (NVENC, QuickSync) for scale
Muxer	Mux video + audio into container (MP4, WebM)	FFmpeg
Storage	Upload recording to object storage	S3, GCS, Azure Blob
Transcoding Pipeline	Generate multiple resolutions/formats for playback	Async job queue (SQS + workers)
Indexing	Extract metadata, generate thumbnail, speech-to-text	ML service (AWS Transcribe, Google Speech-to-Text)

Recording Flow

sequenceDiagram participant U as User participant W as Web UI participant API as Meeting API participant SFU as SFU participant REC as Recording Service participant S3 as Object Storage participant TRANSCODE as Transcoding Pipeline U->>W: Click "Record" W->>API: POST /meetings/{id}/record API->>SFU: Enable recording tap for room Note over SFU,REC: SFU forwards media to recorder SFU->>REC: RTP streams (video + audio) loop During Meeting REC->>REC: Decode streams REC->>REC: Composite grid layout REC->>REC: Encode to H.264 REC->>REC: Write to temporary file end U->>W: End meeting or stop recording W->>API: POST /meetings/{id}/stop-record API->>SFU: Disable recording tap SFU->>REC: Stop signal REC->>REC: Finalize video file (MP4) REC->>S3: Upload raw recording REC->>API: Notify recording ready API->>TRANSCODE: Queue transcoding job TRANSCODE->>S3: Download raw recording TRANSCODE->>TRANSCODE: Generate 1080p, 720p, 480p TRANSCODE->>TRANSCODE: Extract audio TRANSCODE->>TRANSCODE: Generate thumbnails TRANSCODE->>S3: Upload transcoded files TRANSCODE->>API: Transcoding complete API->>W: Recording available W->>U: Show playback link

Composition Strategies

Layout	Description	Use Case
Active Speaker	Show only current speaker full-screen	Webinars, presentations
Gallery Grid	Show all participants in equal-sized tiles (2x2, 3x3, etc.)	Team meetings, default
Spotlight	Large tile for presenter, small tiles for others	Keynotes, demos
Shared Screen Focus	Screen share in main area, participant thumbnails on side	Screen sharing sessions
Audio-Only	Static image or slides + audio track	Podcasts, interviews

Recording Performance Optimization

Optimization	Technique	Benefit
Hardware Encoding	Use GPU encoders (NVENC, QuickSync)	10-20x faster than CPU, lower cost per stream
Dedicated Recording Servers	Separate fleet for recording (high CPU/GPU, lower count)	Don't impact SFU performance
Async Transcoding	Upload raw recording immediately, transcode offline	Reduce real-time resource pressure
Adaptive Composition	Don't decode/compose participants not visible in layout	Save CPU on large meetings
Lower Recording Quality	Record at 720p/1080p even if participants send higher	Reduce storage, encoding cost

9. Screen Sharing

Screen Sharing Requirements

Aspect	Requirement	Rationale
Resolution	1080p or native display (up to 4K)	Text clarity, fine details
Frame Rate	5-15 fps typical, up to 30fps for motion	Static content doesn't need high FPS
Bitrate	1-3 Mbps (higher for motion content like videos)	Balance quality and bandwidth
Codec	H.264 or VP8 with content optimization	Text/UI encoding tuning
Latency	< 500ms end-to-end	Interactive demos, cursor tracking

Screen Sharing Architecture

Component	Responsibility	Details
Capture API	Capture screen/window pixels	WebRTC getDisplayMedia() API, native SDK
Separate Stream	Screen share as independent WebRTC track	Independent from camera video, different encoding params
Content Hint	Encoder optimization for screen content	contentHint = "detail" (prioritize sharpness over motion)
SFU Forwarding	SFU forwards screen share to all participants	Higher priority than camera video
Client Rendering	Display screen share in main area, camera videos as thumbnails	Automatic layout switch

Content-Optimized Encoding

Content Type	Encoding Settings	Rationale
Text/Slides	High spatial quality, low FPS (5-10), high keyframe interval	Static content, prioritize sharpness
Video Playback	Higher FPS (30), lower spatial quality acceptable	Motion content, prioritize smoothness
Code/Terminal	High sharpness, monospace font rendering, low FPS	Text clarity critical
CAD/Design Tools	High resolution, adaptive FPS (5-30)	Detail + motion when panning

Screen Sharing Optimizations

Optimization	Technique	Benefit
Motion Detection	Analyze screen content, increase FPS during motion	Save bandwidth when static
Region of Interest	Encode only changed regions (differential encoding)	Lower bitrate for mostly-static screens
Downscale on Receiver	Send 1080p, receivers with low bandwidth request 720p simulcast layer	Adaptive quality
Audio Mixing	Mix system audio (app sounds) with screen share	Share video/audio playback
Cursor Overlay	Capture cursor position, render on receiver side	Avoid encoding cursor into every frame
Automatic Pause	Detect screen lock/screensaver, pause screen share	Privacy + bandwidth savings

Screen Share + Camera Layout

10. Scalability & Global Architecture

Regional Edge Architecture

Layer	Component	Location	Purpose
Edge	STUN/TURN Servers	50+ PoPs globally	Minimize NAT traversal latency
Edge	Regional SFUs	10-20 regions	Low-latency media routing
Regional	Signaling Servers	Each region	WebSocket connections for control
Regional	Recording Services	Each region	Reduce media backhaul for recording
Central	API Gateway	Multi-region (active-active)	Meeting management, auth
Central	Databases	Primary + replicas	User data, meeting metadata
Central	Object Storage	Multi-region replication	Recordings, assets

Room Placement Strategy

Strategy	How It Works	Use Case
First-Join Region	Place meeting in region of first participant	Simple, works for same-region meetings
Geographic Centroid	Calculate centroid of all participants, pick nearest region	Optimize for average latency across participants
Majority Region	Place in region with most participants	Optimize for majority, acceptable latency for minority
Cascading Multi-Region	SFU in each participant region, cascade to master	Global meetings with good latency for all
Dynamic Migration	Migrate room if participant distribution changes significantly	Long meetings where geography shifts

SFU Auto-Scaling

Metric	Threshold	Action
CPU Utilization	> 70%	Scale up (add SFU instances)
Network Throughput	> 80% of NIC capacity	Scale up
Active Rooms	> 1000 rooms per SFU	Scale up
Room Load Balancing	New room request	Assign to least-loaded SFU in region
Predictive Scaling	Time-of-day patterns (9am meetings spike)	Pre-scale before peak hours
Graceful Shutdown	During scale-down	Drain connections (no new rooms), wait for existing to end

Global Deployment Architecture

graph TB subgraph "US-East Region" US_STUN[STUN/TURN
us-east-1] US_SFU[SFU Pool
100s instances] US_SIG[Signaling
WebSocket] US_REC[Recording
Service] end subgraph "EU-West Region" EU_STUN[STUN/TURN
eu-west-1] EU_SFU[SFU Pool
100s instances] EU_SIG[Signaling
WebSocket] EU_REC[Recording
Service] end subgraph "APAC Region" APAC_STUN[STUN/TURN
ap-southeast-1] APAC_SFU[SFU Pool
100s instances] APAC_SIG[Signaling
WebSocket] APAC_REC[Recording
Service] end subgraph "Global Services" LB[Global Load Balancer
GeoDNS] API[API Gateway
Multi-region] AUTH[Auth Service] DB[(PostgreSQL
Primary + Replicas)] REDIS[(Redis
Cluster)] KAFKA[Kafka
Event Stream] end subgraph "Storage" S3[S3 Multi-Region
Recordings] end US_SIG & EU_SIG & APAC_SIG --> LB LB --> API API --> AUTH API --> DB API --> REDIS US_SFU -.cascade.-> EU_SFU EU_SFU -.cascade.-> APAC_SFU US_REC --> S3 EU_REC --> S3 APAC_REC --> S3 US_SFU & EU_SFU & APAC_SFU --> KAFKA style LB fill:#FFD700 style API fill:#FFD700

Availability & Fault Tolerance

Component	Failure Mode	Mitigation
SFU Instance	Instance crash/network failure	Client reconnects to different SFU (brief interruption), room state in Redis
Entire SFU Region	Regional outage	Clients reconnect to next-closest region (higher latency but service continues)
Signaling Server	WebSocket connection lost	Client auto-reconnects, exponential backoff, session resume
Database	Primary failure	Automatic failover to replica (< 30s downtime), read replicas for scale
Redis	Cache miss or failure	Rebuild room state from DB (slower but functional)
Recording Service	Instance failure during recording	Retry recording on different instance, partial recording saved
Network Partition	Inter-region link down	Cascading SFU fails over to alternative path, or operate isolated

Monitoring & Observability

Metric	What to Measure	Alert Threshold
Packet Loss	% of RTP packets lost per stream	> 5% sustained loss
RTT (Round-Trip Time)	Latency between client and SFU	> 200ms consistently
Jitter	Variation in packet arrival time	> 50ms
Connection Success Rate	% of users successfully joining meetings	< 98%
SFU CPU/Memory	Resource utilization per SFU instance	> 80% sustained
Time to First Frame	Latency from join to first video render	> 3 seconds
Active Meetings	Concurrent meeting count	Track for capacity planning
TURN Usage %	% of connections using TURN relay	> 30% (indicates NAT traversal issues)

Interview Tips & Key Callouts

Start with Clarifications

Question	Why It Matters
"What's the max participants per meeting?"	1:1 can be P2P, 10+ needs SFU, 1000+ needs cascading
"Is recording required?"	Adds complex server-side composition, storage costs
"Do we need to support 1000 viewers or 1000 active speakers?"	1000 viewers → webinar mode (SFU + CDN), 1000 speakers → massive complexity
"What about screen sharing?"	Separate high-res stream, content-optimized encoding
"Are most users in one region or global?"	Single region → simple, global → need cascading SFUs

Critical Design Decisions

Decision	Recommendation	Justification
P2P vs SFU?	P2P for 1:1, SFU for 2+	P2P doesn't scale beyond 2-3 (O(N²) connections), SFU scales linearly
SFU vs MCU?	SFU strongly preferred	Lower cost, better quality, easier scaling, client flexibility
Simulcast or SVC?	Simulcast (broader support)	SVC more efficient but limited browser/device support
WebRTC or custom?	WebRTC	Browser native, proven NAT traversal, encrypted by default
Where to place SFUs?	Edge (close to users)	Minimize latency for real-time media
Recording architecture?	Server-side composition	Consistent output, no client dependency, easier compliance

Deep Dive Topics (if time permits)

Topic	Key Points
NAT Traversal	ICE framework, STUN for discovery, TURN for relay (~20% of connections), symmetric NAT is hardest
Bandwidth Estimation	GCC algorithm, monitor packet loss + RTT, probe for capacity, adapt bitrate dynamically
Cascading SFUs	Regional SFUs for local participants, master SFU aggregates cross-region, reduces inter-region bandwidth
Simulcast	Client sends multiple qualities, SFU selects per receiver, essential for large meetings
Gallery View Optimization	Last-N active speakers, pagination, downgrade off-screen tiles, cap at ~49 visible streams
Security	DTLS-SRTP for media encryption, E2EE option (client-side encryption, SFU can't decrypt)
Recording Composition	FFmpeg for decode/compose/encode, hardware encoders (NVENC), layout engine, async transcoding

Common Pitfalls to Avoid

Pitfall	Why It's Wrong	Correct Approach
Using HTTP for media	Too much latency, overhead	WebRTC over UDP (SRTP)
Full mesh for group calls	O(N²) connections, doesn't scale past 4-5 participants	Use SFU (O(N) per participant)
Single global SFU	High latency for distant users	Regional SFUs at edge
Ignoring NAT traversal	~20% of users behind symmetric NAT, connections will fail	Must have TURN servers
Fixed bitrate	Network varies, fixed rate causes packet loss or underutilization	Dynamic bitrate adaptation (GCC)
Sending all streams in large meetings	Bandwidth explosion (49 streams * 1.5 Mbps = 73 Mbps)	Last-N, simulcast, gallery pagination

Impressive Points to Mention

Point	Impact
"TURN is expensive (2x bandwidth), optimize STUN success rate"	Shows cost awareness
"Use hardware encoders (NVENC) for recording to reduce cost 10-20x"	Practical optimization knowledge
"Simulcast upload is higher but essential for selective forwarding quality"	Understands tradeoffs
"For 1000+ webinars, use SFU for presenters + CDN (HLS) for viewers"	Knows when to hybrid approaches
"Cascading SFUs reduce cross-region bandwidth by aggregating locally first"	Advanced scaling technique
"Gallery view caps at ~49 streams (7x7), use pagination or last-N beyond that"	Practical client-side limits
"Screen share needs content hint 'detail' for text clarity, lower FPS acceptable"	Domain-specific optimization

Scaling Numbers Summary

Metric	Value
Daily Participants	300 million
Peak Concurrent Users	~34 million
Peak Concurrent Meetings	~7 million
Peak Infrastructure Bandwidth	~54 Tbps ingress, ~218 Tbps egress
Daily Recording Storage	~4.7 PB (with 5% recording rate)
User Upload Bandwidth	1.5-3 Mbps (720p-1080p video)
User Download Bandwidth	1.5-7.5 Mbps (1-5 streams in gallery)