Large-Scale Chat System Design
A scalable chat architecture designed around persistent WebSocket connections, distributed message delivery, offline message handling, presence tracking, asynchronous processing, and independent media delivery.
Scale & Capacity
- Registered Users: 1B
- Daily Active Users: 500M
- Concurrent Connections: 50M
- Messages: 10B/day
- Average Throughput: ~115K messages/sec
- Peak Throughput: ~350K–500K messages/sec
Message Delivery
Online Users:
- Client sends a message through a persistent WebSocket connection
- Chat server generates a unique
message_id - Redis resolves the recipient's active chat server
- Message is forwarded directly to the recipient's connection
- Delivery acknowledgment is returned
- Message is asynchronously queued for analytics and archival
Offline Users:
- Presence state determines whether the recipient is connected
- Undelivered messages are stored using an inbox backed by Redis ZSET and persistent storage
- Push notification is triggered
- Message is queued for asynchronous processing
- Pending messages are retrieved when the recipient reconnects
- Messages are delivered through the new WebSocket connection
Distributed Architecture
Connection Layer:
- Load balancers distribute persistent WebSocket connections across chat servers
- Chat servers scale horizontally as connection volume increases
- Service discovery tracks dynamically available chat servers
- Health checks remove unhealthy instances from active routing
Messaging Layer:
- Redis maintains connection and presence information
- Message queues decouple real-time delivery from asynchronous workloads
- Persistent storage provides durable message and inbox data
- Separate online and offline delivery paths prevent disconnected users from blocking real-time traffic
Media Architecture
Direct Upload Flow:
Client → Presigned Upload URL → Blob Storage → CDN
- Client requests a presigned upload URL
- Media is uploaded directly to blob storage
- Chat servers process only the message and media reference
- Recipient receives the media reference
- CDN handles high-volume media delivery independently from the messaging infrastructure
Fault Tolerance & Scalability
- Horizontal Scaling: Chat servers scale independently to handle connection and message growth
- Connection Distribution: Load balancing prevents individual chat servers from becoming connection bottlenecks
- Failure Detection: Health checks identify unhealthy instances and prevent new connections from being routed to them
- Asynchronous Processing: Queues isolate analytics, archival, notifications, and other background workloads
- Service Discovery: Enables dynamic routing as chat server instances are added or removed
- Decoupled Media: Large media payloads bypass chat servers and are delivered through storage and CDN infrastructure
Capacity Planning
The design uses back-of-the-envelope calculations to reason about:
- Message throughput and peak traffic
- Concurrent WebSocket connections
- Network bandwidth requirements
- Message storage growth
- Media storage requirements
- Horizontal capacity for chat servers
- Redis and queue workloads
System Components
- Load Balancer: Distributes WebSocket connections
- Chat Servers: Maintain persistent connections and process messages
- Redis: Presence, connection routing, and offline inbox coordination
- Message Queue: Asynchronous processing and decoupling
- Database: Durable message and inbox storage
- Presence Service: Tracks user availability
- Notification Service: Handles push notifications for offline users
- Blob Storage: Direct media storage
- CDN: Globally distributed media delivery
- Service Discovery: Dynamic service and instance management