← Back to DevBytes
Kubernetes July 03, 2026 6 min read

Designing a Chat System on Kubernetes

Introduction: What is a Chat System on Kubernetes?

A chat system enables real-time, bidirectional communication between users. On Kubernetes, such a system is decomposed into microservices that are deployed, scaled, and managed as containerized workloads. The typical architecture includes:

Kubernetes orchestrates these components, providing automated scaling, self-healing, and rolling updates – critical for production-grade chat applications.

Why It Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Designing a chat system on Kubernetes offers several advantages:

Real‑world examples include Slack’s internal microservices, Discord’s containerized backend, and open‑source platforms like Mattermost.

How to Design and Deploy a Chat System on Kubernetes

Architecture Overview

We will build a minimal but functional chat system with the following components:

Messages flow: Client → WebSocket Server → Redis Pub/Sub → all WebSocket Server instances → broadcast to connected clients.

Prerequisites

Step 1: Setting Up the Message Broker (Redis)

We’ll deploy a single‑node Redis instance using a Deployment and a Service. For production, consider a Redis Cluster or use the Bitnami Helm chart.

# redis-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7-alpine
        ports:
        - containerPort: 6379
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "128Mi"
            cpu: "200m"
---
# redis-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: redis
spec:
  selector:
    app: redis
  ports:
    - protocol: TCP
      port: 6379
      targetPort: 6379

Apply with kubectl apply -f redis-deployment.yaml -f redis-service.yaml.

Step 2: Deploying the WebSocket Server

We’ll create a simple Node.js server that subscribes to Redis Pub/Sub and broadcasts messages to all connected clients. First, the application code (Dockerfile and server.js):

# Dockerfile
FROM node:18-alpine
WORKDIR /app
COPY package.json .
RUN npm install
COPY server.js .
EXPOSE 8080
CMD ["node", "server.js"]

// server.js
const WebSocket = require('ws');
const redis = require('redis');

const PORT = process.env.PORT || 8080;
const REDIS_HOST = process.env.REDIS_HOST || 'redis';

const wss = new WebSocket.Server({ port: PORT });
const publisher = redis.createClient({ url: `redis://${REDIS_HOST}:6379` });
const subscriber = redis.createClient({ url: `redis://${REDIS_HOST}:6379` });

publisher.connect();
subscriber.connect();

subscriber.subscribe('chat', (message) => {
  // Broadcast to all connected clients
  wss.clients.forEach((client) => {
    if (client.readyState === WebSocket.OPEN) {
      client.send(message);
    }
  });
});

wss.on('connection', (ws) => {
  ws.on('message', (data) => {
    const msg = data.toString();
    // Publish to Redis – all server instances receive it via subscriber
    publisher.publish('chat', msg);
  });
  ws.send('Welcome to the chat!');
});

console.log(`WebSocket server running on port ${PORT}`);

// package.json
{
  "name": "chat-server",
  "version": "1.0.0",
  "dependencies": {
    "ws": "^8.16.0",
    "redis": "^4.6.10"
  }
}

Build the Docker image: docker build -t chat-server:latest . and push to your registry (e.g., Docker Hub). Then create the Kubernetes Deployment and Service:

# chat-server-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: chat-server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: chat-server
  template:
    metadata:
      labels:
        app: chat-server
    spec:
      containers:
      - name: chat-server
        image: your-registry/chat-server:latest
        ports:
        - containerPort: 8080
        env:
        - name: REDIS_HOST
          value: "redis"
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "128Mi"
            cpu: "200m"
---
# chat-server-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: chat-server
spec:
  selector:
    app: chat-server
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  # Important for WebSocket persistence across replicas
  sessionAffinity: ClientIP

Apply the manifests.

Step 3: Configuring Ingress for WebSocket

To expose the chat server externally, we need an Ingress that supports WebSocket upgrades. The NGINX Ingress Controller works well. Create the following Ingress resource:

# chat-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: chat-ingress
  annotations:
    nginx.ingress.kubernetes.io/websocket-services: "chat-server"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  ingressClassName: nginx
  rules:
  - host: chat.example.com
    http:
      paths:
      - path: /ws
        pathType: Prefix
        backend:
          service:
            name: chat-server
            port:
              number: 8080

Replace chat.example.com with your domain. Apply it, then ensure DNS resolves to the Ingress controller’s external IP.

Step 4: Scaling with Horizontal Pod Autoscaler (HPA)

To automatically adjust the number of WebSocket server replicas based on CPU usage, create an HPA:

# chat-server-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: chat-server-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: chat-server
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60

Apply: kubectl apply -f chat-server-hpa.yaml.

Step 5: Persisting Chat History (Optional)

To store chat logs, deploy a PostgreSQL StatefulSet with a PersistentVolumeClaim. This ensures each pod gets stable storage. Example:

# postgres-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: "postgres"
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15-alpine
        env:
        - name: POSTGRES_PASSWORD
          value: "changeme"
        - name: POSTGRES_DB
          value: "chatdb"
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: postgres-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi
---
# postgres-service.yaml (headless)
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  selector:
    app: postgres
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
  clusterIP: None

Your WebSocket server can then insert messages into the database for retrieval.

Best Practices

Conclusion

Designing a chat system on Kubernetes is a powerful approach to building a scalable, resilient real‑time communication platform. By breaking down the application into WebSocket servers, a message broker, and a database, and by leveraging Kubernetes features such as Deployments, Services, StatefulSets, HPAs, and Ingresses, you can create a production‑ready system that handles thousands of concurrent connections with ease. The architecture outlined here serves as a solid foundation – you can extend it with authentication, rate limiting, message persistence, and more sophisticated routing. With careful attention to best practices like sticky sessions, health probes, and monitoring, your chat system will be both robust and maintainable.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles