Chapter 52: Design Dropbox/Google Drive

Distributed File Storage and Synchronization System

1. Requirements

Functional Requirements

Requirement	Description	Priority
Upload/Download Files	Users can upload files from any device and download to any device	P0
Cross-Device Sync	Files automatically sync across all user devices in real-time	P0
File Sharing	Share files/folders with other users or via public links	P0
File Versioning	Keep history of file changes, allow rollback to previous versions	P1
Offline Access	Access and edit files offline, sync when connection restored	P1
Conflict Detection	Detect and resolve conflicts when same file edited on multiple devices	P0

Non-Functional Requirements

Requirement	Target	Reasoning
High Availability	99.99% uptime	Users need reliable access to files
Strong Consistency	Eventual consistency acceptable	Sync delays tolerable, but must converge
Low Latency	<500ms for metadata ops	Fast navigation and file operations
Durability	99.999999999% (11 9s)	Zero data loss acceptable
Scalability	500M users, 1B files/day	Massive scale required
Bandwidth Efficiency	Only sync changed blocks	Minimize network usage, especially mobile

Scale Parameters

Metric	Value
Total Users	500 million
Daily Active Users (DAU)	100 million (20%)
Files Uploaded Per Day	1 billion
Average File Size	1 MB
Devices Per User	3 (phone, laptop, tablet)
Read:Write Ratio	10:1

2. Capacity Estimation

Storage Capacity

Component	Calculation	Result
Daily Upload Volume	1B files × 1 MB/file	1 PB/day
Yearly Upload Volume	1 PB/day × 365 days	365 PB/year
5-Year Storage (raw)	365 PB/year × 5 years	1,825 PB = 1.8 EB
Deduplication Savings	40% reduction (typical)	-730 PB
5-Year Storage (deduped)	1,825 PB × 0.6	1,095 PB = 1.1 EB
Replication Factor	3× for durability	3.3 EB total

Metadata Storage

Item	Count	Size per Item	Total
Users	500M	1 KB	500 GB
Files (5 years)	1.8 trillion	500 bytes	900 TB
File Chunks	7.2 trillion (4 chunks/file avg)	100 bytes	720 TB
Versions (3 avg/file)	5.4 trillion	200 bytes	1,080 TB
Total Metadata			2.7 PB

Bandwidth

Operation	Calculation	Result
Upload (writes)	1 PB/day ÷ 86,400 sec	11.6 GB/s
Download (reads)	11.6 GB/s × 10 (read:write ratio)	116 GB/s
Total Bandwidth	Upload + Download	127.6 GB/s
Peak Factor (3×)	127.6 GB/s × 3	383 GB/s

QPS (Queries Per Second)

Operation	Daily Count	QPS	Peak QPS (3×)
File Uploads	1B	11,574	34,722
File Downloads	10B	115,740	347,220
Metadata Operations	50B (list, stat, etc.)	578,700	1,736,100
Sync Notifications	5B (change events)	57,870	173,610

3. High-Level Design

System Architecture

graph TB subgraph "Client Layer" Desktop[Desktop Client
Windows/Mac/Linux] Mobile[Mobile Apps
iOS/Android] Web[Web Browser
React SPA] end subgraph "Edge Layer" CDN[CDN
CloudFront] LB[Load Balancer
ALB] end subgraph "API Layer" API1[API Server 1] API2[API Server 2] API3[API Server N] end subgraph "Service Layer" Upload[Upload Service
Chunking] Download[Download Service
Reassembly] Sync[Sync Service
Change Detection] Notify[Notification Service
WebSocket/Long Poll] Share[Sharing Service
ACL Management] end subgraph "Storage Layer" Meta[(Metadata DB
PostgreSQL
Sharded)] Cache[(Redis Cache
Hot Metadata)] Blocks[Block Storage
S3/GCS
Content-Addressable] end subgraph "Background Jobs" Dedup[Deduplication Worker] Version[Version Manager] GC[Garbage Collector] end Desktop --> CDN Mobile --> CDN Web --> CDN CDN --> LB LB --> API1 & API2 & API3 API1 & API2 & API3 --> Upload & Download & Sync & Notify & Share Upload --> Meta Upload --> Cache Upload --> Blocks Download --> Cache Download --> Meta Download --> Blocks Sync --> Meta Sync --> Notify Share --> Meta Blocks --> Dedup Meta --> Version Blocks --> GC

Component Responsibilities

Component	Responsibility	Technology
Desktop/Mobile Client	Local file watching, chunking, sync coordination, conflict detection	C++/Swift/Kotlin
CDN	Cache popular files at edge, reduce origin load	CloudFront/Akamai
API Gateway/LB	Route requests, rate limiting, authentication	ALB + API Gateway
Upload Service	Receive file chunks, compute hashes, deduplicate, store blocks	Go/Rust
Download Service	Fetch blocks, reassemble files, serve to clients	Go/Rust
Sync Service	Detect changes, compute delta, broadcast to connected devices	Go
Notification Service	Maintain WebSocket/long-poll connections, push change events	Node.js/Go
Metadata DB	Store file tree, user data, permissions, version history	PostgreSQL (sharded)
Cache	Cache hot metadata, recent file listings, active sessions	Redis Cluster
Block Storage	Store deduplicated file chunks, immutable, content-addressable	S3/GCS

Data Flow Summary

Flow	Path	Latency
File Upload	Client → API → Upload Service → Block Storage + Metadata DB	2-5s (depending on file size)
File Download	Client → CDN/API → Download Service → Block Storage	100-500ms (cache hit) / 1-3s (cache miss)
Sync Notification	Upload → Sync Service → Notification Service → WebSocket → Client	100-300ms
Metadata Query	Client → API → Cache/Metadata DB	50-200ms

4. File Chunking

Chunking Strategy

Aspect	Design Decision	Rationale
Chunk Size	4 MB	Balance between dedup granularity and metadata overhead. Too small = excessive chunks, too large = poor dedup
Chunking Algorithm	Fixed-size + Rolling hash (Rabin fingerprint) for small edits	Fixed-size simple and fast. Rolling hash better handles insertions/deletions in middle of file
Hash Function	SHA-256	Content-addressable storage. Hash = block ID. Cryptographically secure prevents collisions
Deduplication Scope	Global (cross-user)	Same block stored only once regardless of how many users have it. Massive storage savings
Compression	Gzip/Zstd before storage	Further reduce storage costs. ~30% additional savings

Content-Addressable Storage

graph LR F["Original File 12 MB"] F --> C1["Chunk 1 4 MB"] F --> C2["Chunk 2 4 MB"] F --> C3["Chunk 3 4 MB"] C1 --> H1["SHA-256 a3f5e9..."] C2 --> H2["SHA-256 b7c4d2..."] C3 --> H3["SHA-256 e8f1a6..."] H1 --> B1[s3://blocks/a3/f5/a3f5e9...] H2 --> B2[s3://blocks/b7/c4/b7c4d2...] H3 --> B3[s3://blocks/e8/f1/e8f1a6...] B1 & B2 & B3 -.-> Meta

Deduplication Savings

Scenario	Without Dedup	With Dedup	Savings
User uploads same file twice	2 × 100 MB = 200 MB	100 MB (second upload only updates metadata)	50%
10 users share company logo	10 × 5 MB = 50 MB	5 MB (9 users point to same blocks)	90%
Small edit to large file	2 × 1 GB = 2 GB	1 GB + 4 MB (only changed chunk uploaded)	~50%
System-wide average	100%	60% (after dedup)	40%

Chunk Metadata Schema

Field	Type	Description
chunk_hash	VARCHAR(64) PK	SHA-256 hash of chunk content
size	INT	Uncompressed size in bytes
compressed_size	INT	Compressed size in bytes
ref_count	INT	Number of files referencing this chunk
storage_path	VARCHAR(255)	S3 key (e.g., s3://blocks/a3/f5/a3f5e9...)
created_at	TIMESTAMP	When chunk first stored

5. Upload Flow

Upload Sequence

sequenceDiagram participant C as Client participant API as API Server participant US as Upload Service participant MD as Metadata DB participant S3 as Block Storage (S3) participant Sync as Sync Service participant N as Notification Service C->>C: 1. Detect file change
(via file system watcher) C->>C: 2. Chunk file (4 MB blocks) C->>C: 3. Compute SHA-256 for each chunk C->>API: 4. POST /files/upload/init
{filename, size, chunks: [hash1, hash2, ...]} API->>MD: 5. Check which chunks exist MD-->>API: 6. Return missing chunk hashes API-->>C: 7. Upload URLs for missing chunks loop For each missing chunk C->>US: 8. PUT /chunks/{hash}
(chunk data) US->>US: 9. Verify hash matches US->>S3: 10. Store chunk S3-->>US: 11. ACK US->>MD: 12. Insert chunk metadata
(or increment ref_count) US-->>C: 13. Chunk uploaded end C->>API: 14. POST /files/upload/commit
{file_id, chunks: [hash1, hash2, ...]} API->>MD: 15. Create file metadata entry
(file_id, path, chunks, version) MD-->>API: 16. File committed API-->>C: 17. Upload complete API->>Sync: 18. File change event
(user_id, file_id, version) Sync->>N: 19. Broadcast to user's devices N->>C: 20. WebSocket: file changed
(other devices sync)

Upload API Design

Endpoint	Method	Purpose	Request	Response
/files/upload/init	POST	Initialize upload, get list of missing chunks	{filename, size, chunks: [hash1, ...]}	{upload_id, missing_chunks: [hash2, hash5, ...], upload_urls: {...}}
/chunks/{hash}	PUT	Upload a single chunk	Binary chunk data	{status: "uploaded", hash}
/files/upload/commit	POST	Finalize upload after all chunks uploaded	{upload_id, file_path, chunks: [...]}	{file_id, version, url}

Upload Optimizations

Optimization	Technique	Benefit
Delta Sync	Only upload chunks that changed (new hash)	For 1 GB file with 4 MB edit, upload 4 MB instead of 1 GB (99.6% savings)
Parallel Upload	Upload multiple chunks concurrently (up to 10 parallel streams)	10× faster for large files on high-bandwidth connections
Resumable Upload	Track which chunks uploaded. On failure, resume from last chunk	Handle flaky connections, don't re-upload completed chunks
Compression	Gzip chunks before upload (if not already compressed format like .jpg)	30-50% bandwidth savings for text/code files
Pre-computation	Compute hashes while file is being written (before upload)	Reduce time to first byte. Start upload immediately when file saved
Batching	Batch multiple small file uploads into single request	Reduce overhead for folders with many small files

Error Handling

Error	Handling
Network failure during upload	Exponential backoff retry. Resume from last successful chunk
Hash mismatch	Reject chunk, client recomputes and re-uploads
Storage quota exceeded	Return 413 Payload Too Large, prompt user to upgrade
Duplicate upload (same file uploaded concurrently)	Use idempotency key (upload_id). Second commit is no-op

6. Synchronization

Sync Architecture

graph TB subgraph "Device 1 (Desktop)" D1[File System Watcher] D1C[Local Metadata Cache] D1S[Sync Client] end subgraph "Device 2 (Phone)" D2[File System Watcher] D2C[Local Metadata Cache] D2S[Sync Client] end subgraph "Device 3 (Laptop)" D3[File System Watcher] D3C[Local Metadata Cache] D3S[Sync Client] end subgraph "Cloud Services" NS[Notification Service
WebSocket Server] SS[Sync Service
Conflict Detector] MD[(Metadata DB
Source of Truth)] end D1 --> D1S D1S <--> NS D1S --> D1C D2 --> D2S D2S <--> NS D2S --> D2C D3 --> D3S D3S <--> NS D3S --> D3C NS <--> SS SS <--> MD D1S -.->|Edit file| SS SS -.->|Notify change| D2S SS -.->|Notify change| D3S

Notification Mechanisms

Mechanism	Protocol	Pros	Cons	Use Case
WebSocket	Full-duplex persistent connection	Real-time, low latency, bi-directional	Hard to scale (many open connections), not HTTP-friendly	Desktop/web clients with stable connection
Long Polling	HTTP GET with long timeout	Works through proxies, HTTP-compatible	Higher latency, more overhead than WebSocket	Mobile clients, corporate networks
Push Notifications	APNs (iOS), FCM (Android)	Works when app backgrounded, battery efficient	Not real-time, delivery not guaranteed	Mobile apps, wake device to sync
Polling	Periodic HTTP GET	Simple, stateless	High latency (poll interval), inefficient	Fallback only

Sync Conflict Detection

Conflict Type	Detection Method	Example
Edit-Edit Conflict	Same file edited on two devices before sync	Device A and Device B both edit doc.txt offline, then come online
Delete-Edit Conflict	One device deletes file while another edits it	Device A deletes file.txt, Device B edits it (both offline)
Move-Edit Conflict	File moved on one device, edited on another	Device A moves file to different folder, Device B edits original path
Folder Rename Conflict	Folder renamed to different names on two devices	Device A renames "Photos" to "Images", Device B renames to "Pictures"

Version Vectors for Conflict Detection

Field	Type	Purpose
file_id	UUID	Unique file identifier (survives renames/moves)
version	INT	Monotonically increasing version number
device_id	UUID	Which device made the change
modified_at	TIMESTAMP	When change was made (for last-write-wins)
parent_version	INT	Previous version this edit was based on
checksum	VARCHAR(64)	Hash of file content (detect conflicts)

Sync Strategies

Strategy	When to Use	Trade-offs
Last-Write-Wins (LWW)	Simple conflicts, user doesn't care about losing edits	Simple but can lose data. Use timestamp to pick winner
Fork (Conflict Copy)	Can't automatically merge, preserve both versions	Safe (no data loss) but creates clutter. User manually merges
Operational Transform (OT)	Real-time collaborative editing (Google Docs style)	Complex to implement, requires operational transforms. Best UX
CRDTs (Conflict-free Replicated Data Types)	Eventual consistency, automatic conflict resolution	Works well for certain data structures (sets, counters). Complex for general files

7. Conflict Resolution

Conflict Resolution Decision Tree

graph TD Start[File Change Detected] --> Check{Check Parent Version} Check -->|Parent = Latest| NoConflict[No Conflict
Apply Change] Check -->|Parent != Latest| Conflict[Conflict Detected] Conflict --> Type{Conflict Type?} Type -->|Both Edit Same Section| Fork[Create Conflict Copy
file.txt & file_conflicted_copy.txt] Type -->|Edit Different Sections| Merge[Auto-Merge
Non-overlapping Changes] Type -->|One Delete, One Edit| DelEdit[Restore Deleted File
Apply Edit] Type -->|One Move, One Edit| MoveEdit[Apply Move & Edit
File at New Location] Fork --> Notify1[Notify User:
Manual Merge Required] Merge --> Verify{Merge
Successful?} Verify -->|Yes| NoConflict Verify -->|No| Fork DelEdit --> Notify2[Notify User:
Deleted File Restored] MoveEdit --> Notify3[Notify User:
File Moved & Updated] NoConflict --> Sync[Sync to All Devices] Notify1 --> Sync Notify2 --> Sync Notify3 --> Sync

Conflict Resolution Rules

Scenario	Rule	Outcome	Example
Same file, different chunks modified	Auto-merge	Combine both changes	Device A edits line 1-10, Device B edits line 50-60. Merge both
Same file, same chunk modified	Fork (create conflict copy)	file.txt + file (conflicted copy).txt	Device A and B both edit line 5. Keep both versions
Delete vs. edit	Edit wins (restore file)	File restored with edits	Device A deletes file.txt, Device B edits it. Restore with edits
Rename conflict	Last-write-wins (by timestamp)	Most recent rename wins	Device A renames to "foo.txt" at 10:00, Device B to "bar.txt" at 10:05. Result: "bar.txt"
Folder delete with files inside	Move files to parent, delete folder	Files preserved, folder removed	Device A deletes /folder, Device B adds /folder/file.txt. Move file.txt to /, delete folder
Binary file conflict	Fork (cannot auto-merge binary)	Keep both versions	Device A and B both edit image.jpg. Create image.jpg + image (conflicted copy).jpg

Conflict Copy Naming

Original File	Conflict Copy Name	Pattern
document.txt	document (conflicted copy 2025-05-12).txt	{name} (conflicted copy {date}).{ext}
report.pdf	report (Bob's conflicted copy 2025-05-12).pdf	{name} ({user}'s conflicted copy {date}).{ext}
Multiple conflicts same day	file (conflicted copy 2025-05-12 2).txt	{name} (conflicted copy {date} {n}).{ext}

Merge Algorithm (Text Files)

Step	Action	Details
1. Fetch Versions	Get base version, version A, version B	Base = common ancestor, A = device 1 changes, B = device 2 changes
2. Compute Diffs	Diff(base → A) and Diff(base → B)	Use line-based diff algorithm (similar to git diff)
3. Detect Overlaps	Check if any chunks modified in both A and B	If no overlap, auto-merge. If overlap, create conflict copy
4. Apply Patches	Apply non-overlapping changes from both A and B to base	Merge result = base + patches from A + patches from B
5. Create New Version	Save merged file as new version, update metadata	Version N+1, parent_versions = [version_A, version_B]

8. Metadata Store

Database Schema

Table	Columns	Indexes	Sharding Key
users	user_id (PK), email, password_hash, storage_quota, storage_used, created_at	email (unique)	user_id
files	file_id (PK), user_id, parent_folder_id, name, path, size, chunks (JSONB), version, checksum, created_at, modified_at, deleted_at	user_id, parent_folder_id, path, checksum	user_id
folders	folder_id (PK), user_id, parent_folder_id, name, path, created_at, modified_at	user_id, parent_folder_id, path	user_id
chunks	chunk_hash (PK), size, compressed_size, ref_count, storage_path, created_at	None (global, not sharded)	N/A
versions	version_id (PK), file_id, user_id, version_number, chunks (JSONB), size, checksum, device_id, created_at	file_id, user_id	user_id
shares	share_id (PK), file_id, owner_user_id, shared_with_user_id, permission (read/write), created_at, expires_at	file_id, owner_user_id, shared_with_user_id	owner_user_id
devices	device_id (PK), user_id, device_name, device_type, last_active_at, created_at	user_id	user_id

files.chunks JSONB Example

file_id	chunks (JSONB)
12345	`[ {"hash": "a3f5e9...", "size": 4194304, "offset": 0}, {"hash": "b7c4d2...", "size": 4194304, "offset": 4194304}, {"hash": "e8f1a6...", "size": 2097152, "offset": 8388608} ]`

Sharding Strategy

Aspect	Strategy	Details
Sharding Key	user_id	All of a user's data on same shard. Enables fast queries for user's files
Sharding Method	Consistent hashing	hash(user_id) % num_shards. Enables easy re-sharding
Number of Shards	Start with 256, grow to 4096+	500M users → ~2M users/shard (at 256 shards)
Cross-Shard Queries	Avoid via denormalization	For shared files, duplicate metadata in both owner's and sharee's shards
Chunks Table	Not sharded (global)	Chunks shared across all users. Use distributed cache (Redis) + eventual consistency

Caching Strategy

Cache Key	Value	TTL	Invalidation
user:{user_id}:files	List of user's file metadata (recent 1000)	5 minutes	On file upload/delete/rename
file:{file_id}	File metadata (name, size, chunks, version)	1 hour	On file update
chunk:{hash}	Chunk metadata (size, storage_path)	24 hours	Never (immutable)
folder:{folder_id}:children	List of files/folders in directory	5 minutes	On child add/remove
user:{user_id}:quota	storage_used, storage_quota	1 minute	On file upload/delete (eventual consistency OK)

9. Block Storage

Object Storage Design

Aspect	Design Choice	Rationale
Storage Backend	AWS S3 / Google Cloud Storage	Proven durability (11 9s), auto-replication, cost-effective at exabyte scale
Storage Class	S3 Standard for recent blocks, S3 Infrequent Access for old versions	Optimize cost. Recent files accessed frequently, old versions rarely
Bucket Organization	Single bucket per region, shard by hash prefix	s3://dropbox-blocks-us-east-1/a3/f5/a3f5e9... (first 2 bytes of hash for prefix)
Compression	Gzip for text, skip for already-compressed (jpg, mp4, zip)	30% storage savings for compressible files. Detect MIME type
Encryption	AES-256 server-side encryption (SSE-S3)	Encryption at rest required for compliance. S3 handles key management
Replication	S3 automatically replicates within region (3 copies)	S3 handles durability. For disaster recovery, cross-region replication to 1 other region

Storage Path Structure

Hash (SHA-256)	S3 Key	URL
a3f5e9c7d1...	a3/f5/a3f5e9c7d1...	s3://dropbox-blocks-us-east-1/a3/f5/a3f5e9c7d1...
b7c4d2a8e6...	b7/c4/b7c4d2a8e6...	s3://dropbox-blocks-us-east-1/b7/c4/b7c4d2a8e6...
e8f1a6c3b9...	e8/f1/e8f1a6c3b9...	s3://dropbox-blocks-us-east-1/e8/f1/e8f1a6c3b9...

Storage Lifecycle

Stage	Age	Storage Class	Cost (per GB/month)
Hot	0-30 days	S3 Standard	$0.023
Warm	31-90 days	S3 Infrequent Access	$0.0125
Cold	91-365 days	S3 Glacier Flexible	$0.004
Archive	365+ days	S3 Glacier Deep Archive	$0.00099

Garbage Collection

Trigger	Action	Safety Mechanism
chunk.ref_count = 0	Mark chunk for deletion	Grace period: 30 days before actual deletion
User deletes file	Decrement ref_count for each chunk in file	File soft-deleted (deleted_at timestamp). Recoverable for 30 days
Daily GC job	Scan chunks with ref_count=0 and marked_for_deletion > 30 days ago	Only delete if no references found in backup DB sweep
Version purge	Delete versions older than retention policy (e.g., 90 days)	Always keep latest version. Decrement ref_count for purged chunks

Cost Optimization

Technique	Savings	Implementation
Deduplication	40% storage reduction	Content-addressable storage. Same chunk stored once
Compression	30% additional reduction (text files)	Gzip chunks before storing. Skip already-compressed formats
Lifecycle Policies	60% cost reduction for old data	Move infrequently accessed data to Glacier
Intelligent Tiering	Auto-optimize based on access patterns	S3 Intelligent-Tiering automatically moves objects between tiers
Aggressive Version Pruning	Reduce version storage overhead	Keep all versions < 30 days, then 1/week for 90 days, then 1/month forever

10. Sharing & Permissions

Permission Levels

Level	Permissions	Use Case
Owner	Full control: read, write, delete, share, change permissions	File creator, has all rights
Editor	Read, write, comment. Cannot delete or share	Team members collaborating on document
Commenter	Read and add comments. Cannot edit	Reviewers providing feedback
Viewer	Read-only. Can download but not edit	Sharing final version with stakeholders
No Access	Cannot see file exists	Revoked access

Sharing Methods

Method	Implementation	Pros	Cons
Direct Share (by email)	Add entry to shares table: (file_id, shared_with_user_id, permission)	Secure, trackable, granular permissions	Requires recipient to have account
Public Link	Generate random token, create link: dropbox.com/s/{token}	Anyone with link can access, no account needed	Less secure, no audit trail of who accessed
Team Folder	Shared folder with group permissions. All team members auto-granted access	Easy to manage for organizations	Coarse-grained (all or nothing for folder)
Password-Protected Link	Public link + password requirement	Balance between convenience and security	Password can be shared out of band

ACL (Access Control List) Schema

Field	Type	Description
acl_id	UUID PK	Unique ACL entry ID
resource_id	UUID	file_id or folder_id
resource_type	ENUM(file, folder)	What kind of resource
principal_id	UUID	user_id or group_id who has access
principal_type	ENUM(user, group, public)	Type of principal
permission	ENUM(owner, editor, commenter, viewer)	Permission level
granted_by	UUID	user_id who granted this permission
created_at	TIMESTAMP	When access granted
expires_at	TIMESTAMP (nullable)	Optional expiration for time-limited sharing

Public Link Sharing

Field	Value
Link Format	https://dropbox.com/s/{token} where token is random 22-char alphanumeric
Token Generation	Cryptographically secure random (128-bit entropy)
Token Mapping	Store in share_links table: (token → file_id, permission, password_hash, expires_at)
Access Control	When user accesses link, lookup token, check expiration, verify password if required
Revocation	Delete token from share_links table. Link immediately becomes invalid
Analytics	Track link views, downloads, IP addresses (for abuse detection)

Sharing Flow

Step	User Action	System Action
1	Click "Share" on file	Show share dialog with options
2	Enter recipient email or choose "Get Link"	For email: lookup user_id. For link: generate token
3	Select permission level (viewer/editor)	Validate owner has permission to grant this level
4	Click "Share"	Insert ACL entry, send email notification to recipient
5	Recipient clicks link in email	Add file to recipient's "Shared with me" folder
6	Recipient opens file	Check ACL, serve file if permitted

11. Scalability

Scale Bottlenecks & Solutions

Bottleneck	Symptom	Solution
Metadata DB write throughput	Slow file uploads, timeouts on commit	Shard by user_id. Use async write-behind for non-critical metadata. Cache-aside pattern
Block storage bandwidth	Slow downloads, upload congestion	CDN for hot files. Edge caching. Parallel chunk download
Notification fanout	Slow sync, delayed notifications	Pub/sub with Kafka. Partition by user_id. Batching for bulk changes
Metadata query latency	Slow folder listing, search	Redis cache for hot paths. Elasticsearch for full-text search. Pre-compute folder sizes
WebSocket connection limits	Can't handle 100M concurrent connections	Stateless notification servers. Use long polling as fallback. Conn limits per server: 10K, scale to 10K servers

Deduplication Impact

Metric	Without Dedup	With Dedup (40% savings)	Impact
5-Year Storage	1,825 PB	1,095 PB	730 PB savings
Storage Cost (5 years)	$503M (at $0.023/GB/mo avg)	$302M	$201M savings
Bandwidth (upload)	11.6 GB/s	6.96 GB/s (only new chunks)	40% reduction
User Experience	Upload 1 GB file	Upload 600 MB (40% already exists)	2.4× faster upload

Edge Caching Strategy

Content Type	Cache Location	TTL	Cache Hit Rate
Popular files (top 1%)	CDN (CloudFront)	24 hours	90%
Recently uploaded (< 1 day)	Regional cache (Redis)	1 hour	70%
User's own files	Local client cache	Until modified	95%
Old/cold files	S3 origin (no cache)	N/A	0%

Incremental Sync Savings

Scenario	Full Sync	Incremental Sync	Data Transferred
1 MB file, change 1 line	1 MB	4 KB (1 chunk)	0.4% of full sync
100 MB video, edit metadata	100 MB	4 KB (first chunk with metadata)	0.004% of full sync
1 GB file, append 5 MB	1.005 GB	8 MB (2 new chunks)	0.8% of full sync
Folder with 1000 files, change 1	Sync all 1000 files	Sync only changed chunks of 1 file	99.9% reduction

Database Scaling Milestones

Users	DB Strategy	Shards	Storage per Shard
1M - 10M	Single primary + read replicas	1	5 TB
10M - 100M	Shard by user_id	16	30 TB
100M - 500M	Shard by user_id + read replicas per shard	256	10 TB
500M+	Shard by user_id + multi-region + tiered storage	4096	650 GB

Performance Targets

Operation	P50 Latency	P99 Latency	Throughput
List files in folder	50 ms	200 ms	1M QPS
Upload 1 MB file	500 ms	2 sec	50K uploads/sec
Download 1 MB file (cache hit)	100 ms	500 ms	500K downloads/sec
Sync notification delivery	100 ms	500 ms	100K notifications/sec
Search files	200 ms	1 sec	100K searches/sec

Interview Callouts

Key Discussion Points

Topic	What to Emphasize	Why It Matters
File Chunking	4 MB chunks, SHA-256 hashing, content-addressable storage	Enables deduplication and incremental sync. Core innovation of Dropbox
Deduplication	Global dedup (cross-user), 40% storage savings	Makes system economically viable at scale. Differentiator vs naive approach
Sync Protocol	WebSocket for real-time, long polling fallback, version vectors for conflict detection	Hard problem: keep N devices in sync with offline support and conflicts
Metadata Sharding	Shard by user_id, co-locate user's data, avoid cross-shard queries	Scalability bottleneck. Single DB can't handle 500M users
Conflict Resolution	Auto-merge non-overlapping, fork on overlap, last-write-wins for moves	UX critical. Poor conflict handling = data loss or user frustration
Upload Optimization	Delta sync (only changed chunks), parallel upload, resumable	Network bandwidth is scarce (especially mobile). Optimize for minimal data transfer
Block Storage	S3 for durability, lifecycle policies for cost, CDN for hot files	Exabyte-scale storage requires cloud object storage. Cost optimization critical
Sharing & Permissions	ACLs for granular control, public links with tokens, expiration support	Security model. Balance between usability (easy sharing) and security (don't leak data)

Follow-up Questions to Anticipate

Question	Answer Direction
"How do you handle a user uploading the same file twice?"	Compute hash, check if chunks exist. If yes, only update metadata (no storage overhead). Second upload is nearly instant
"What if two users edit the same file at the same time?"	Detect via version vectors. If chunks overlap, create conflict copy. If different sections, auto-merge. Notify both users
"How do you prevent a user from exceeding their storage quota?"	Check quota before upload. Reject if would exceed. Track storage_used in user table, update on upload/delete (eventually consistent OK)
"What if S3 goes down?"	Serve from CDN cache (cache hit). For cache miss, show error, retry with exponential backoff. Multi-region S3 replication for DR
"How do you handle a file that's being edited while syncing?"	File watcher detects write. Wait for file to stabilize (no writes for 1 sec). Then chunk and upload. If edited mid-upload, restart upload
"How do you optimize for mobile devices with limited bandwidth?"	Delta sync (only changed chunks). Compress chunks. Prioritize small files. Background sync when on WiFi. Defer large uploads
"What's your database schema for the file tree?"	files table (file_id, parent_folder_id, name, chunks JSONB). Adjacency list for tree. Path column for fast subtree queries. Shard by user_id
"How do you detect and delete orphaned chunks?"	ref_count on chunks table. When file deleted, decrement ref_count. GC job finds chunks with ref_count=0 > 30 days, deletes from S3
"How do you scale WebSocket connections to 100M users?"	Stateless WS servers (10K connections each = 10K servers). Load balancer routes by user_id hash. Fallback to long polling. Use Kafka for fanout
"How do you handle versioning?"	versions table (file_id, version_number, chunks, created_at). Keep all versions < 30 days, then prune. UI shows version history, restore button

Design Trade-offs to Discuss

Trade-off	Option A	Option B	Chosen	Rationale
Chunk Size	Small (256 KB)	Large (16 MB)	Medium (4 MB)	Balance: small = better dedup but more metadata overhead. Large = less overhead but worse dedup for edits
Sync Notification	WebSocket (real-time)	Polling (simple)	WebSocket + long poll fallback	Real-time UX important. But polling needed for corporate networks that block WS
Conflict Resolution	Last-write-wins (simple)	Fork (safe)	Hybrid: auto-merge if possible, else fork	LWW loses data. Fork always works but creates clutter. Auto-merge when safe is best UX
Deduplication Scope	Per-user only	Global (cross-user)	Global	Global gives massive savings (40%). Privacy concern addressed by content-addressable storage (no file names leaked)
Metadata Consistency	Strong consistency	Eventual consistency	Eventual consistency	File systems tolerate slight lag. Strong consistency too expensive at scale. Sync within 1 sec is acceptable

Back-of-Envelope Math to Show

Calculation	Work	Result
Storage for 500M users, 20 GB each	500M × 20 GB = 10 EB (raw)	With 40% dedup: 6 EB
Metadata for 1.8 trillion files	1.8T files × 500 bytes/file = 900 TB	Easily fits in sharded PostgreSQL
Bandwidth for 1B uploads/day	1B × 1 MB / 86,400 sec = 11.6 GB/s	Need ~12 GB/s upload capacity
WebSocket connections for 100M DAU	100M connections / 10K per server = 10K servers	Need 10K stateless WS servers
Cost savings from deduplication	730 PB saved × $0.023/GB/mo × 12 mo = $201M/year	Dedup pays for itself many times over