Designing a Chat System for 100 Million Users
What It Is: A Hyper-Scale Real-Time Messaging Architecture
Designing a chat system for 100 million users means building a distributed, fault-tolerant, real-time messaging platform capable of handling hundreds of millions of concurrent connections, trillions of messages per day, and sub-100-millisecond delivery latency across the globe. This is not a simple client-server app — it is a multi-tiered, horizontally scalable system that combines WebSocket gateways, message queues, distributed databases, caching layers, and presence services into a cohesive architecture.
A chat system at this scale must support:
- Massive concurrency — 100M+ active users with persistent TCP/WebSocket connections
- Reliable delivery — Exactly-once or at-least-once message delivery semantics
- Low latency — End-to-end delivery under 200ms for 99th percentile
- Global distribution — Users in every timezone, with data locality and regional failover
- Multi-device sync — Seamless experience across phone, desktop, and web
- Rich features — Read receipts, typing indicators, reactions, file sharing, group chats
Why It Matters: The Economics and Engineering of Scale
A poorly designed chat system at 100M users collapses under its own weight. Connection overhead, message fan-out, database write amplification, and state synchronization become existential challenges. Getting the architecture right matters because:
- User retention — Every millisecond of latency increases churn by measurable percentages at scale
- Infrastructure cost — Inefficient fan-out can cost millions in unnecessary server and bandwidth bills
- Operational complexity — Without proper partitioning and replication, incidents become catastrophic
- Competitive advantage — WhatsApp, Telegram, and Discord handle this scale; your system must too
The core challenge is the fan-out problem: when one user sends a message to a group of 100,000 members, the system must deliver that message to every connected member with minimal latency while not overwhelming any single server or network link.
How to Use It: Building the Core Architecture
We will walk through the complete architecture layer by layer, with working code examples for each critical component. The system uses a regional fan-out with global coordination pattern.
1. Connection Gateway Layer (WebSocket Servers)
The entry point for all clients. Each gateway server maintains hundreds of thousands of persistent WebSocket connections. Gateways are stateless regarding message storage but stateful regarding active connections.
// Node.js WebSocket Gateway Server
const WebSocket = require('ws');
const Redis = require('ioredis');
const { v4: uuidv4 } = require('uuid');
const PORT = process.env.PORT || 8080;
const REDIS_URL = process.env.REDIS_URL || 'redis://localhost:6379';
const pub = new Redis(REDIS_URL);
const sub = new Redis(REDIS_URL);
// Maps userId -> set of WebSocket connections (multi-device support)
const userConnections = new Map();
// Maps connectionId -> { ws, userId, deviceId }
const connectionMap = new Map();
const wss = new WebSocket.Server({ port: PORT, maxPayload: 1024 * 1024 });
wss.on('connection', (ws, req) => {
const connectionId = uuidv4();
ws.on('message', async (data) => {
try {
const msg = JSON.parse(data);
switch (msg.type) {
case 'auth':
// Authenticate and register the connection
const { userId, deviceId, token } = msg.payload;
const isValid = await verifyToken(userId, token);
if (!isValid) {
ws.send(JSON.stringify({ type: 'error', payload: 'Unauthorized' }));
ws.close();
return;
}
// Register connection
if (!userConnections.has(userId)) {
userConnections.set(userId, new Set());
}
userConnections.get(userId).add(connectionId);
connectionMap.set(connectionId, { ws, userId, deviceId });
// Subscribe to user's personal channel for incoming messages
sub.subscribe(`user:${userId}`, (err) => {
if (err) console.error('Subscribe error:', err);
});
ws.send(JSON.stringify({ type: 'auth_ok', payload: { connectionId } }));
break;
case 'message':
// Handle outbound message from client
await handleOutboundMessage(msg.payload, connectionId);
break;
case 'ping':
ws.send(JSON.stringify({ type: 'pong' }));
break;
}
} catch (err) {
console.error('Message handler error:', err);
ws.send(JSON.stringify({ type: 'error', payload: 'Internal error' }));
}
});
ws.on('close', () => {
// Clean up connection
const conn = connectionMap.get(connectionId);
if (conn) {
const userSockets = userConnections.get(conn.userId);
if (userSockets) {
userSockets.delete(connectionId);
if (userSockets.size === 0) {
userConnections.delete(conn.userId);
sub.unsubscribe(`user:${conn.userId}`);
}
}
connectionMap.delete(connectionId);
}
});
ws.on('error', (err) => {
console.error('WebSocket error:', err);
});
});
// Listen for incoming messages from other gateways via Redis Pub/Sub
sub.on('message', (channel, message) => {
const userId = channel.replace('user:', '');
const connections = userConnections.get(userId);
if (!connections) return;
const parsed = JSON.parse(message);
for (const connId of connections) {
const conn = connectionMap.get(connId);
if (conn && conn.ws.readyState === WebSocket.OPEN) {
conn.ws.send(JSON.stringify({
type: 'new_message',
payload: parsed
}));
}
}
});
async function verifyToken(userId, token) {
// In production, validate JWT or session token against auth service
return token.startsWith('valid_');
}
async function handleOutboundMessage(payload, connectionId) {
const conn = connectionMap.get(connectionId);
if (!conn) return;
const message = {
id: uuidv4(),
senderId: conn.userId,
conversationId: payload.conversationId,
content: payload.content,
contentType: payload.contentType || 'text',
timestamp: Date.now()
};
// Publish to the message relay service
await pub.publish('message_relay', JSON.stringify(message));
}
console.log(`Gateway server running on port ${PORT}`);
2. Message Relay & Fan-Out Service
This service receives messages from gateways and determines which users should receive them. For 1:1 chats, the fan-out is trivial. For group chats with millions of members, we use a two-phase fan-out with membership sharding.
// Message Relay Service (Go)
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"time"
"github.com/go-redis/redis/v8"
"github.com/google/uuid"
)
type Message struct {
ID string `json:"id"`
SenderID string `json:"senderId"`
ConversationID string `json:"conversationId"`
Content string `json:"content"`
ContentType string `json:"contentType"`
Timestamp int64 `json:"timestamp"`
}
type RelayService struct {
redisClient *redis.Client
membershipCache *MembershipCache
messageStore *MessageStore
}
func NewRelayService(redisURL string) *RelayService {
rdb := redis.NewClient(&redis.Options{
Addr: redisURL,
PoolSize: 1000,
})
return &RelayService{
redisClient: rdb,
membershipCache: NewMembershipCache(rdb),
messageStore: NewMessageStore(rdb),
}
}
func (rs *RelayService) Start(ctx context.Context) {
pubsub := rs.redisClient.Subscribe(ctx, "message_relay")
defer pubsub.Close()
ch := pubsub.Channel()
for msg := range ch {
var message Message
if err := json.Unmarshal([]byte(msg.Payload), &message); err != nil {
log.Printf("Error unmarshaling message: %v", err)
continue
}
// Process message asynchronously
go rs.processMessage(ctx, &message)
}
}
func (rs *RelayService) processMessage(ctx context.Context, msg *Message) {
start := time.Now()
// Step 1: Store the message
if err := rs.messageStore.StoreMessage(ctx, msg); err != nil {
log.Printf("Error storing message: %v", err)
return
}
// Step 2: Get conversation members
// For groups with > 10K members, this uses a sharded membership service
members, err := rs.membershipCache.GetConversationMembers(ctx, msg.ConversationID)
if err != nil {
log.Printf("Error getting members: %v", err)
return
}
// Step 3: Fan-out to all members except sender
for _, memberID := range members {
if memberID == msg.SenderID {
continue
}
// Publish to user's personal channel
// Each user channel is consumed by all gateway servers
userChannel := fmt.Sprintf("user:%s", memberID)
deliveryPayload, _ := json.Marshal(map[string]interface{}{
"message": msg,
"type": "delivery",
"seq": time.Now().UnixNano(),
})
err := rs.redisClient.Publish(ctx, userChannel, string(deliveryPayload)).Err()
if err != nil {
log.Printf("Error publishing to %s: %v", userChannel, err)
// Queue for retry via dead-letter queue
rs.queueForRetry(ctx, memberID, msg)
}
}
// Step 4: Update conversation metadata
rs.redisClient.ZAdd(ctx, fmt.Sprintf("conversation:%s:messages", msg.ConversationID),
&redis.Z{Score: float64(msg.Timestamp), Member: msg.ID})
elapsed := time.Since(start)
log.Printf("Message %s fanned out to %d recipients in %v",
msg.ID, len(members)-1, elapsed)
}
func (rs *RelayService) queueForRetry(ctx context.Context, userID string, msg *Message) {
data, _ := json.Marshal(map[string]interface{}{
"userId": userID,
"message": msg,
})
rs.redisClient.LPush(ctx, "retry_queue", string(data))
}
// MembershipCache with sharded lookups for large groups
type MembershipCache struct {
redisClient *redis.Client
}
func NewMembershipCache(rdb *redis.Client) *MembershipCache {
return &MembershipCache{redisClient: rdb}
}
func (mc *MembershipCache) GetConversationMembers(ctx context.Context, convID string) ([]string, error) {
// For conversations with < 10K members, use a sorted set
// For larger groups, fan out to membership shards
members, err := mc.redisClient.SMembers(ctx, fmt.Sprintf("conv:%s:members", convID)).Result()
if err == redis.Nil {
// Fetch from database and cache
return mc.loadMembersFromDB(ctx, convID)
}
return members, err
}
func (mc *MembershipCache) loadMembersFromDB(ctx context.Context, convID string) ([]string, error) {
// In production, this queries a sharded membership database
// For demonstration, return sample data
members := []string{"user_1", "user_2", "user_3"}
mc.redisClient.SAdd(ctx, fmt.Sprintf("conv:%s:members", convID), members)
mc.redisClient.Expire(ctx, fmt.Sprintf("conv:%s:members", convID), time.Hour)
return members, nil
}
func main() {
ctx := context.Background()
relay := NewRelayService("localhost:6379")
log.Println("Relay service started")
relay.Start(ctx)
}
3. Message Store with Time-Based Partitioning
At 100M users, a single database cannot hold all messages. We use time-based partitioning with a hybrid Cassandra/PostgreSQL approach. Messages are stored in monthly partitions, with hot data in memory and cold data in compressed columnar storage.
// Message Store with Time-Based Partitioning (Go)
package main
import (
"context"
"fmt"
"time"
"github.com/gocql/gocql"
"github.com/jackc/pgx/v4/pgxpool"
)
type MessageStore struct {
cassandra *gocql.Session
postgres *pgxpool.Pool
}
type StoredMessage struct {
MessageID string `json:"message_id"`
ConversationID string `json:"conversation_id"`
SenderID string `json:"sender_id"`
Content string `json:"content"`
ContentType string `json:"content_type"`
CreatedAt time.Time `json:"created_at"`
}
func NewMessageStore(cassandra *gocql.Session, pg *pgxpool.Pool) *MessageStore {
return &MessageStore{
cassandra: cassandra,
postgres: pg,
}
}
// partitionKey returns the monthly partition for a given timestamp
func partitionKey(t time.Time) string {
return fmt.Sprintf("messages_%04d_%02d", t.Year(), t.Month())
}
func (ms *MessageStore) StoreMessage(ctx context.Context, msg *Message) error {
partition := partitionKey(time.Unix(0, msg.Timestamp))
// Store in Cassandra for fast time-range queries
cqlQuery := fmt.Sprintf(`
INSERT INTO %s (message_id, conversation_id, sender_id, content, content_type, created_at)
VALUES (?, ?, ?, ?, ?, ?)
`, partition)
err := ms.cassandra.Query(cqlQuery,
msg.ID,
msg.ConversationID,
msg.SenderID,
msg.Content,
msg.ContentType,
time.Unix(0, msg.Timestamp),
).WithContext(ctx).Exec()
if err != nil {
return fmt.Errorf("cassandra write failed: %w", err)
}
// Also store in PostgreSQL for complex queries and full-text search
pgQuery := `
INSERT INTO messages (message_id, conversation_id, sender_id, content, content_type, created_at)
VALUES ($1, $2, $3, $4, $5, $6)
ON CONFLICT (message_id) DO NOTHING
`
_, err = ms.postgres.Exec(ctx, pgQuery,
msg.ID,
msg.ConversationID,
msg.SenderID,
msg.Content,
msg.ContentType,
time.Unix(0, msg.Timestamp),
)
return err
}
func (ms *MessageStore) GetMessages(ctx context.Context, convID string,
before time.Time, limit int) ([]StoredMessage, error) {
partition := partitionKey(before)
query := fmt.Sprintf(`
SELECT message_id, conversation_id, sender_id, content, content_type, created_at
FROM %s
WHERE conversation_id = ? AND created_at < ?
ORDER BY created_at DESC
LIMIT ?
`, partition)
iter := ms.cassandra.Query(query, convID, before, limit).WithContext(ctx).Iter()
var messages []StoredMessage
var msg StoredMessage
for iter.Scan(&msg.MessageID, &msg.ConversationID, &msg.SenderID,
&msg.Content, &msg.ContentType, &msg.CreatedAt) {
messages = append(messages, msg)
}
if err := iter.Close(); err != nil {
return nil, fmt.Errorf("cassandra read failed: %w", err)
}
return messages, nil
}
// Create schema for monthly partitions
func (ms *MessageStore) CreatePartition(year int, month int) error {
partition := partitionKey(time.Date(year, time.Month(month), 1, 0, 0, 0, 0, time.UTC))
query := fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS %s (
message_id uuid PRIMARY KEY,
conversation_id text,
sender_id text,
content text,
content_type text,
created_at timestamp
) WITH CLUSTERING ORDER BY (created_at DESC)
AND compaction = {'class': 'TimeWindowCompactionStrategy'}
AND default_time_to_live = 7776000 -- 90 days for hot data
`, partition)
return ms.cassandra.Query(query).Exec()
}
4. Presence Service with Heartbeat Aggregation
Tracking which of 100M users are online requires a highly efficient presence system. We use a Redis-backed heartbeat aggregation with regional sharding and TTL-based expiry.
// Presence Service (Go)
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"sync"
"time"
"github.com/go-redis/redis/v8"
)
type PresenceService struct {
redisClient *redis.Client
region string
// Local cache of presence states for fast reads
localCache sync.Map
}
type PresenceState struct {
UserID string `json:"userId"`
Status string `json:"status"` // "online", "away", "offline"
Device string `json:"device"`
LastSeen int64 `json:"lastSeen"`
Region string `json:"region"`
}
func NewPresenceService(redisURL string, region string) *PresenceService {
rdb := redis.NewClient(&redis.Options{
Addr: redisURL,
PoolSize: 500,
MinIdleConns: 100,
})
return &PresenceService{
redisClient: rdb,
region: region,
}
}
// Heartbeat receives periodic pings from connected users
func (ps *PresenceService) Heartbeat(ctx context.Context, userID string, device string) error {
state := PresenceState{
UserID: userID,
Status: "online",
Device: device,
LastSeen: time.Now().Unix(),
Region: ps.region,
}
data, _ := json.Marshal(state)
// Store in Redis with 60-second TTL
// Each heartbeat resets the TTL
key := fmt.Sprintf("presence:%s", userID)
err := ps.redisClient.Set(ctx, key, string(data), 60*time.Second).Err()
if err != nil {
return err
}
// Update local cache
ps.localCache.Store(userID, state)
// Publish presence update for real-time subscribers
ps.redisClient.Publish(ctx, "presence_updates", string(data))
return nil
}
// GetPresence returns the current state for a user
func (ps *PresenceService) GetPresence(ctx context.Context, userID string) (*PresenceState, error) {
// Check local cache first
if cached, ok := ps.localCache.Load(userID); ok {
state := cached.(PresenceState)
// If cached within last 30 seconds, return it
if time.Now().Unix()-state.LastSeen < 30 {
return &state, nil
}
}
// Fall back to Redis
key := fmt.Sprintf("presence:%s", userID)
data, err := ps.redisClient.Get(ctx, key).Result()
if err == redis.Nil {
return &PresenceState{UserID: userID, Status: "offline", LastSeen: 0}, nil
}
if err != nil {
return nil, err
}
var state PresenceState
if err := json.Unmarshal([]byte(data), &state); err != nil {
return nil, err
}
// Update local cache
ps.localCache.Store(userID, state)
return &state, nil
}
// GetBulkPresence returns presence for up to 1000 users at once (pipeline)
func (ps *PresenceService) GetBulkPresence(ctx context.Context, userIDs []string) (map[string]*PresenceState, error) {
results := make(map[string]*PresenceState)
// Batch check local cache first
var missIDs []string
for _, uid := range userIDs {
if cached, ok := ps.localCache.Load(uid); ok {
state := cached.(PresenceState)
if time.Now().Unix()-state.LastSeen < 30 {
results[uid] = &state
continue
}
}
missIDs = append(missIDs, uid)
}
if len(missIDs) == 0 {
return results, nil
}
// Pipeline Redis gets for cache misses
pipe := ps.redisClient.Pipeline()
cmds := make(map[string]*redis.StringCmd)
for _, uid := range missIDs {
key := fmt.Sprintf("presence:%s", uid)
cmds[uid] = pipe.Get(ctx, key)
}
_, err := pipe.Exec(ctx)
if err != nil && err != redis.Nil {
log.Printf("Pipeline error: %v", err)
}
for uid, cmd := range cmds {
data, err := cmd.Result()
if err == redis.Nil {
results[uid] = &PresenceState{UserID: uid, Status: "offline", LastSeen: 0}
continue
}
if err != nil {
log.Printf("Error fetching presence for %s: %v", uid, err)
continue
}
var state PresenceState
if err := json.Unmarshal([]byte(data), &state); err != nil {
continue
}
ps.localCache.Store(uid, state)
results[uid] = &state
}
return results, nil
}
// CleanupExpired runs periodically to remove stale entries from local cache
func (ps *PresenceService) CleanupExpired(ctx context.Context) {
ticker := time.NewTicker(30 * time.Second)
for {
select {
case <-ticker.C:
ps.localCache.Range(func(key, value interface{}) bool {
state := value.(PresenceState)
if time.Now().Unix()-state.LastSeen > 120 {
ps.localCache.Delete(key)
}
return true
})
case <-ctx.Done():
ticker.Stop()
return
}
}
}
func main() {
ctx := context.Background()
svc := NewPresenceService("localhost:6379", "us-east-1")
go svc.CleanupExpired(ctx)
// Example heartbeat
svc.Heartbeat(ctx, "user_42", "iphone_15")
state, _ := svc.GetPresence(ctx, "user_42")
log.Printf("User %s is %s", state.UserID, state.Status)
}
5. Message Synchronization with Vector Clocks
For multi-device sync and offline delivery, we implement vector clock-based message ordering. Each device maintains its own clock, and the server uses these clocks to determine message ordering and detect conflicts.
// Vector Clock Sync Service (Go)
package main
import (
"context"
"fmt"
"sync"
"time"
"github.com/go-redis/redis/v8"
)
type VectorClock struct {
DeviceClocks map[string]int64 `json:"deviceClocks"`
mu sync.RWMutex
}
type SyncService struct {
redisClient *redis.Client
clockStore *sync.Map // conversationID -> *VectorClock
}
func NewSyncService(redisURL string) *SyncService {
rdb := redis.NewClient(&redis.Options{
Addr: redisURL,
})
return &SyncService{
redisClient: rdb,
clockStore: &sync.Map{},
}
}
// SyncRequest represents a client asking for new messages
type SyncRequest struct {
UserID string `json:"userId"`
ConversationID string `json:"conversationId"`
DeviceID string `json:"deviceId"`
LastKnownClock map[string]int64 `json:"lastKnownClock"` // Device clocks from client
MaxMessages int `json:"maxMessages"`
}
// SyncResponse returns new messages and updated clocks
type SyncResponse struct {
Messages []*StoredMessage `json:"messages"`
NewClock map[string]int64 `json:"newClock"`
HasMore bool `json:"hasMore"`
SyncToken string `json:"syncToken"`
}
func (ss *SyncService) PerformSync(ctx context.Context, req *SyncRequest) (*SyncResponse, error) {
// Get the conversation's current vector clock
clock := ss.getOrCreateClock(req.ConversationID)
clock.mu.RLock()
defer clock.mu.RUnlock()
// Determine which messages this device has missed
// by comparing the device's last known clock with the conversation clock
missedDevices := make([]string, 0)
for deviceID, serverTime := range clock.DeviceClocks {
clientTime, exists := req.LastKnownClock[deviceID]
if !exists || clientTime < serverTime {
missedDevices = append(missedDevices, deviceID)
}
}
// Also check if the requesting device itself has new messages
if clientTime, exists := req.LastKnownClock[req.DeviceID]; !exists ||
clientTime < clock.DeviceClocks[req.DeviceID] {
missedDevices = append(missedDevices, req.DeviceID)
}
if len(missedDevices) == 0 {
// Device is fully synced
return &SyncResponse{
Messages: []*StoredMessage{},
NewClock: req.LastKnownClock,
HasMore: false,
SyncToken: fmt.Sprintf("%d", time.Now().UnixNano()),
}, nil
}
// Fetch missed messages from the message store
// In production, this queries Cassandra with the device-specific partition
messages, err := ss.fetchMissedMessages(ctx, req.ConversationID,
req.LastKnownClock, req.MaxMessages)
if err != nil {
return nil, fmt.Errorf("fetch missed messages failed: %w", err)
}
// Return the current server clock so the client can update its state
serverClockCopy := make(map[string]int64)
for k, v := range clock.DeviceClocks {
serverClockCopy[k] = v
}
hasMore := len(messages) >= req.MaxMessages
return &SyncResponse{
Messages: messages,
NewClock: serverClockCopy,
HasMore: hasMore,
SyncToken: fmt.Sprintf("%d_%d", time.Now().UnixNano(), len(messages)),
}, nil
}
func (ss *SyncService) getOrCreateClock(conversationID string) *VectorClock {
clockInterface, _ := ss.clockStore.LoadOrStore(conversationID, &VectorClock{
DeviceClocks: make(map[string]int64),
})
return clockInterface.(*VectorClock)
}
// AdvanceClock is called after a message is delivered to update the vector clock
func (ss *SyncService) AdvanceClock(conversationID string, deviceID string) {
clock := ss.getOrCreateClock(conversationID)
clock.mu.Lock()
defer clock.mu.Unlock()
clock.DeviceClocks[deviceID] = time.Now().UnixNano()
}
func (ss *SyncService) fetchMissedMessages(ctx context.Context, convID string,
clientClocks map[string]int64, max int) ([]*StoredMessage, error) {
// Query messages from the partition that are newer than the oldest device clock
var oldestClock int64 = time.Now().UnixNano()
for _, clock := range clientClocks {
if clock < oldestClock {
oldestClock = clock
}
}
// In production, this queries Cassandra with:
// SELECT * FROM messages_{partition} WHERE conversation_id = ? AND created_at > ?
// For this example, return empty
return []*StoredMessage{}, nil
}
// OfflineQueue stores messages for users who are disconnected
type OfflineQueue struct {
redisClient *redis.Client
}
func NewOfflineQueue(redisURL string) *OfflineQueue {
rdb := redis.NewClient(&redis.Options{
Addr: redisURL,
})
return &OfflineQueue{redisClient: rdb}
}
func (oq *OfflineQueue) Enqueue(ctx context.Context, userID string, message *Message) error {
data, _ := json.Marshal(message)
key := fmt.Sprintf("offline:%s", userID)
// Use a sorted set with timestamp as score for ordered replay
return oq.redisClient.ZAdd(ctx, key, &redis.Z{
Score: float64(message.Timestamp),
Member: string(data),
}).Err()
}
func (oq *OfflineQueue) DequeueAll(ctx context.Context, userID string) ([]*Message, error) {
key := fmt.Sprintf("offline:%s", userID)
// Get and remove all queued messages atomically
pipe := oq.redisClient.Pipeline()
rangeCmd := pipe.ZRangeByScore(ctx, key, &redis.ZRangeBy{
Min: "-inf",
Max: "+inf",
})
pipe.Del(ctx, key)
_, err := pipe.Exec(ctx)
if err != nil {
return nil, err
}
results := rangeCmd.Val()
messages := make([]*Message, len(results))
for i, data := range results {
var msg Message
json.Unmarshal([]byte(data), &msg)
messages[i] = &msg
}
return messages, nil
}
6. Global Load Balancer with Geo-Aware Routing
At 100M users, DNS-based load balancing is insufficient. We implement a geo-aware Anycast + HTTP/2 load balancer that routes users to the nearest regional cluster.
// Geo-Aware Router Configuration (NGINX + Lua)
// This config sits at the edge of each regional data center
events {
worker_connections 1000000;
use epoll;
multi_accept on;
}
http {
# Upstream gateway pools per region
upstream gateways_us_east {
least_conn;
server 10.0.1.1:8080 max_fails=3 fail_timeout=10s;
server 10.0.1.2:8080 max_fails=3 fail_timeout=10s;
server 10.0.1.3:8080 max_fails=3 fail_timeout=10s;
keepalive 1000;
}
upstream gateways_eu_west {
least_conn;
server 10.0.2.1:8080 max_fails=3 fail_timeout=10s;
server 10.0.2.2:8080 max_fails=3 fail_timeout=10s;
keepalive 1000;
}
upstream gateways_ap_south {
least_conn;
server 10.0.3.1:8080 max_fails=3 fail_timeout=10s;
server 10.0.3.2:8080 max_fails=3 fail_timeout=10s;
server 10.0.3.3:8080 max_fails=3 fail_timeout=10s;
keepalive 1000;
}
# Lua script for geo-aware routing
lua_shared_dict geo_cache 10m;
init_worker_by_lua_block {
-- Load GeoIP database
local geo = require("resty.maxminddb")
ok, err = geo.init("/etc/nginx/geoip/GeoLite2-City.mmdb")
if not ok then
ngx.log(ngx.ERR, "Failed to load GeoIP: ", err)
end
}
server {
listen 443 ssl http2;
server_name chat.example.com;
ssl_certificate /etc/nginx/ssl/cert.pem;
ssl_certificate_key /etc/nginx/ssl/key.pem;
# WebSocket upgrade support
location /ws {
proxy_pass http://gateways_us_east;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket timeout settings
proxy_read_timeout 86400s;
proxy_send_timeout 86400s;
# Buffering off for real-time
proxy_buffering off;
proxy_request_buffering off;
# Geo-aware routing via Lua
set $backend_region "us_east";
access_by_lua_block {
local client_ip = ngx.var.remote_addr
local geo = require("resty.maxminddb")
-- Look up client's continent/region
local res, err = geo.lookup(client_ip)
if res then
local continent = res.continent and res.continent.code
if continent == "EU" then
ngx.var.backend_region = "eu