Deploying Agents with Docker Compose: Production Patterns

Deploying Agents with Docker Compose: Production Patterns

When autonomous agents move from prototype to production, their deployment architecture becomes critically important. Docker Compose provides a powerful, declarative way to orchestrate multi-container agent systems that are reproducible, scalable, and maintainable. This tutorial covers proven production patterns for deploying LLM-powered agents, task workers, and supporting infrastructure using Docker Compose.

What This Pattern Actually Is

At its core, deploying agents with Docker Compose means packaging each agent component—whether it's a LangChain agent, a custom GPT-based worker, a message queue consumer, or a vector database—into its own Docker container and using a single docker-compose.yml file to define how they interact. This approach treats agents as independent services with defined interfaces, shared networks, and managed lifecycles.

A typical production agent stack includes:

• Agent Runners — Containers executing LLM reasoning loops, tool calls, and decision logic
• Message Brokers — Redis, RabbitMQ, or Kafka for asynchronous task distribution
• State Stores — PostgreSQL, Redis, or dedicated memory services for conversation history and agent state
• Tool Servers — HTTP-based microservices that agents call for specific capabilities (web search, code execution, API integrations)
• Observability Stack — Prometheus, Grafana, and log aggregators for monitoring agent behavior

Why Docker Compose for Production Agents

Running agents directly on a VM or server is tempting for simplicity, but it quickly leads to dependency conflicts, inconsistent environments, and difficult rollbacks. Docker Compose addresses these production pain points directly:

• Reproducible Environments — Every agent runs in an identical container with pinned dependencies, eliminating "works on my machine" issues that plague LLM pipelines with their fast-changing library ecosystems
• Resource Isolation — Memory-hungry agents (especially those loading large models or performing vector operations) get dedicated CPU and memory limits, preventing one runaway agent from starving others
• Declarative Networking — Services communicate through Compose-defined networks with DNS-based discovery, making it trivial to swap agent backends or add new tool servers
• Graceful Scaling — While Docker Compose itself doesn't auto-scale, its service definitions make it straightforward to scale specific agent workers using docker compose up --scale behind a load balancer
• Infrastructure-as-Code — The entire agent topology lives in version control, enabling CI/CD pipelines, review processes, and audit trails

Core Architecture Patterns

Let's explore the most common production patterns, each with a complete working example.

Pattern 1: The Agent + Tool Server Pattern

In this pattern, a reasoning agent container orchestrates tasks by calling out to dedicated tool server containers. This is the simplest production-ready pattern and works well for teams getting started.

# docker-compose.yml
version: '3.9'

services:
  # The main reasoning agent
  agent-runner:
    build:
      context: ./agent
      dockerfile: Dockerfile
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - TOOL_SEARCH_URL=http://search-tool:8080/search
      - TOOL_CODE_URL=http://code-executor:8081/execute
      - REDIS_URL=redis://state-store:6379
    depends_on:
      search-tool:
        condition: service_healthy
      code-executor:
        condition: service_healthy
      state-store:
        condition: service_started
    restart: unless-stopped
    volumes:
      - agent-logs:/app/logs

  # Tool: Web Search microservice
  search-tool:
    build: ./tools/search-tool
    environment:
      - BRAVE_API_KEY=${BRAVE_API_KEY}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped

  # Tool: Sandboxed code execution
  code-executor:
    build: ./tools/code-executor
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped
    # Security: no unrestricted network access
    networks:
      - agent-network
      - no-internet

  # Shared state store for conversation memory
  state-store:
    image: redis:7-alpine
    volumes:
      - redis-data:/data
    restart: unless-stopped

volumes:
  agent-logs:
  redis-data:

networks:
  agent-network:
  no-internet:
    driver: bridge
    internal: true

The agent container's Dockerfile keeps things clean and reproducible:

# ./agent/Dockerfile
FROM python:3.12-slim

WORKDIR /app

# Pin exact versions for reproducibility
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

# Non-root user for security
RUN adduser --disabled-password agentuser
USER agentuser

CMD ["python", "-u", "main.py"]

Pattern 2: The Async Worker Pool Pattern

For high-throughput agent systems, synchronous HTTP calls between containers create bottlenecks. The async worker pool pattern decouples task submission from execution using a message broker. This is the pattern to use when you need to process hundreds or thousands of agent tasks concurrently.

# docker-compose.yml
version: '3.9'

services:
  # API gateway receives tasks from users
  api-gateway:
    build: ./api
    ports:
      - "8000:8000"
    environment:
      - CELERY_BROKER_URL=redis://message-broker:6379/0
      - CELERY_RESULT_BACKEND=redis://message-broker:6379/1
    depends_on:
      message-broker:
        condition: service_healthy
    restart: unless-stopped

  # Celery worker pool - scales horizontally
  agent-worker:
    build: ./worker
    environment:
      - CELERY_BROKER_URL=redis://message-broker:6379/0
      - CELERY_RESULT_BACKEND=redis://message-broker:6379/1
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - TOOL_REGISTRY_URL=http://tool-registry:8080
    depends_on:
      message-broker:
        condition: service_healthy
      tool-registry:
        condition: service_healthy
    restart: unless-stopped
    # Scale this service for more throughput
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '2'
          memory: 2G

  # Central message broker
  message-broker:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - broker-data:/data
    restart: unless-stopped

  # Tool registry for dynamic tool discovery
  tool-registry:
    build: ./tool-registry
    ports:
      - "8080:8080"
    volumes:
      - ./tool-definitions:/definitions
    restart: unless-stopped

  # Monitoring with Flower for Celery tasks
  flower:
    image: mher/flower:latest
    ports:
      - "5555:5555"
    environment:
      - CELERY_BROKER_URL=redis://message-broker:6379/0
    depends_on:
      - message-broker

volumes:
  broker-data:

Here's a production-ready Celery worker implementation:

# ./worker/main.py
import os
import logging
from celery import Celery
from celery.signals import worker_process_init

# Configure structured logging for production
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s | %(levelname)s | %(name)s | %(message)s'
)
logger = logging.getLogger(__name__)

celery_app = Celery(
    'agent_tasks',
    broker=os.environ['CELERY_BROKER_URL'],
    backend=os.environ['CELERY_RESULT_BACKEND'],
)

# Task routing for different agent types
celery_app.conf.task_routes = {
    'tasks.research.*': {'queue': 'research'},
    'tasks.code_gen.*': {'queue': 'code_gen'},
    'tasks.data_analysis.*': {'queue': 'data_analysis'},
}
celery_app.conf.task_acks_late = True
celery_app.conf.task_reject_on_worker_lost = True

@worker_process_init.connect
def setup_worker_init(**kwargs):
    """Initialize expensive resources once per worker process."""
    logger.info("Worker initializing - loading agent configuration")
    # Load agent configs, warm up caches, etc.

@celery_app.task(bind=True, max_retries=3, default_retry_delay=60)
def execute_agent_task(self, task_payload: dict):
    """Execute an agent task with retry logic."""
    try:
        logger.info(f"Executing task {self.request.id} | type={task_payload['type']}")
        # Agent execution logic here
        result = run_agent_pipeline(task_payload)
        return result
    except Exception as exc:
        logger.error(f"Task {self.request.id} failed: {exc}")
        raise self.retry(exc=exc)

Pattern 3: The Sidecar-Enhanced Agent Pattern

Advanced production deployments often use sidecar containers to offload cross-cutting concerns from the agent container. This keeps agent code focused on reasoning while sidecars handle logging, secret management, and proxy responsibilities.

# docker-compose.yml (excerpt showing sidecar pattern)
services:
  agent-core:
    build: ./agent-core
    environment:
      - OPENAI_BASE_URL=http://localhost:8080/v1  # Route through sidecar
    volumes:
      - shared-logs:/var/log/agent
    depends_on:
      llm-proxy:
        condition: service_healthy

  # Sidecar: LLM proxy with rate limiting & logging
  llm-proxy:
    image: ghcr.io/your-org/llm-proxy:latest
    environment:
      - UPSTREAM_URL=https://api.openai.com
      - RATE_LIMIT_RPM=60
      - LOG_LEVEL=info
      - VAULT_TOKEN_FILE=/vault/token
    volumes:
      - shared-logs:/var/log/proxy
      - vault-token:/vault:ro
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 15s
      timeout: 5s

  # Sidecar: Log forwarder
  log-shipper:
    image: fluent/fluent-bit:latest
    volumes:
      - shared-logs:/logs:ro
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
    restart: unless-stopped

volumes:
  shared-logs:
  vault-token:

The LLM proxy sidecar is a critical production component. Here's a minimal but production-grade implementation:

# llm-proxy/main.py
from flask import Flask, request, Response
import requests
import time
import redis
import logging

app = Flask(__name__)
logger = logging.getLogger(__name__)

# Rate limiting state in Redis
redis_client = redis.Redis(
    host=os.environ.get('REDIS_HOST', 'localhost'),
    decode_responses=True
)

RATE_LIMIT_RPM = int(os.environ.get('RATE_LIMIT_RPM', 60))
UPSTREAM_URL = os.environ['UPSTREAM_URL']

@app.route('/v1/chat/completions', methods=['POST'])
def proxy_chat():
    # Rate limit check
    key = f"ratelimit:{request.remote_addr}"
    current = redis_client.incr(key)
    if current == 1:
        redis_client.expire(key, 60)
    if current > RATE_LIMIT_RPM:
        return Response(
            '{"error":"Rate limit exceeded"}',
            status=429,
            mimetype='application/json'
        )

    # Log request (sanitized)
    payload = request.get_json()
    safe_payload = sanitize_for_logging(payload)
    logger.info(f"Proxying request | model={safe_payload.get('model')} | tokens_est={len(str(safe_payload)) // 4}")

    # Forward to upstream
    start = time.time()
    resp = requests.post(
        f"{UPSTREAM_URL}/chat/completions",
        json=payload,
        headers={'Authorization': request.headers.get('Authorization')},
        timeout=90
    )
    latency = time.time() - start

    logger.info(f"Response | status={resp.status_code} | latency={latency:.2f}s")
    return Response(resp.content, status=resp.status_code, mimetype='application/json')

@app.route('/health', methods=['GET'])
def health():
    return '{"status":"ok"}', 200

Managing State and Memory Across Restarts

Production agents must persist state across container restarts. Here are the patterns for different state types:

# docker-compose.yml fragment for state management
services:
  # Conversation memory in PostgreSQL
  conversation-store:
    image: pgvector/pgvector:pg16  # Supports vector embeddings
    environment:
      POSTGRES_USER: agent
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: agent_memory
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./init-scripts:/docker-entrypoint-initdb.d
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U agent -d agent_memory"]
      interval: 10s
      timeout: 5s
      retries: 5

  # Fast ephemeral cache for agent scratchpad
  agent-cache:
    image: redis:7-alpine
    command: redis-server --maxmemory 512mb --maxmemory-policy allkeys-lru
    volumes:
      - cache-data:/data

  # Long-term vector store
  vector-db:
    image: qdrant/qdrant:latest
    volumes:
      - qdrant-storage:/qdrant/storage
    environment:
      QDRANT__SERVICE__GRPC_PORT: 6334
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:6333/health"]
      interval: 30s

volumes:
  pgdata:
  cache-data:
  qdrant-storage:

Production-Grade Health Checks and Resilience

Health checks are not optional in production. Each service needs a health check appropriate to its role, and agent services should implement graceful shutdown handlers:

# ./agent/main.py - Graceful shutdown
import signal
import asyncio

class AgentRunner:
    def __init__(self):
        self.running = True
        self.current_tasks = set()

    async def run(self):
        while self.running:
            task = await self.fetch_next_task()
            if task:
                task_handle = asyncio.create_task(self.process(task))
                self.current_tasks.add(task_handle)
                task_handle.add_done_callback(self.current_tasks.discard)

    async def shutdown(self, sig):
        print(f"Received signal {sig.name}, draining tasks...")
        self.running = False
        # Wait for in-flight tasks to complete (with timeout)
        if self.current_tasks:
            done, pending = await asyncio.wait(
                self.current_tasks,
                timeout=30.0
            )
            for p in pending:
                p.cancel()
        print("Shutdown complete")

runner = AgentRunner()

def handle_signal(sig):
    asyncio.create_task(runner.shutdown(sig))

loop = asyncio.get_event_loop()
for s in (signal.SIGTERM, signal.SIGINT):
    loop.add_signal_handler(s, lambda s=s: handle_signal(s))

loop.run_until_complete(runner.run())

Secrets Management in Compose

Never hardcode API keys in Compose files. Use Docker secrets (in Swarm) or environment files with proper encryption for Compose-only deployments:

# Production .env file pattern (never commit this)
OPENAI_API_KEY=sk-proj-abc123...
ANTHROPIC_API_KEY=sk-ant-xyz789...
BRAVE_API_KEY=BSA-abc...
DB_PASSWORD=secure-generated-password

# .env.example (commit this instead)
OPENAI_API_KEY=your-openai-key-here
ANTHROPIC_API_KEY=your-anthropic-key-here
BRAVE_API_KEY=your-brave-key-here
DB_PASSWORD=generate-a-strong-password

For Docker Compose in production, reference secrets through environment variables and ensure the .env file is excluded from version control with .gitignore. In Docker Swarm or Kubernetes, use their native secret management instead.

Best Practices Summary

• Pin Everything — Use exact image tags (redis:7.2.4-alpine, not redis:latest) and pinned Python package versions in requirements.txt. LLM library APIs change rapidly; reproducibility matters.
• Implement Health Checks on Every Service — Use depends_on with condition: service_healthy to ensure startup ordering. Agents should not start until their tool servers and state stores are ready.
• Set Resource Limits — Use deploy.resources.limits to prevent memory leaks from crashing the host. Agent containers running LLM calls can consume unpredictable amounts of memory.
• Use Non-Root Users — Every Dockerfile should switch to a non-root user for the runtime. This limits blast radius if an agent executes malicious tool output.
• Log to stdout, Not Files — Write logs to stdout/stderr so Docker's logging drivers can capture them. Use structured JSON logging for easy parsing by log aggregators.
• Separate Configuration from Code — Use environment variables for runtime configuration (API endpoints, keys, feature flags) and keep them out of container images.
• Implement Graceful Shutdown — Agents processing tasks need time to finish in-flight work before shutdown. Handle SIGTERM properly with drain patterns.
• Network Segmentation — Put internet-facing tool containers on restricted networks. Code execution sandboxes should not have unrestricted outbound access.
• Version Your Compose Files — Tag your docker-compose.yml with a version in comments and store it in git alongside the agent code it orchestrates.
• Test Restart Scenarios — Regularly test what happens when individual containers restart. State stores should recover; agent workers should resume cleanly.

Putting It All Together: A Complete Production Stack

Here is a full production-ready Compose file that combines the patterns above into a cohesive agent deployment:

# docker-compose.prod.yml
version: '3.9'

services:
  # Ingress: handles user requests, authentication, rate limiting
  gateway:
    image: nginx:1.25-alpine
    ports:
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
    depends_on:
      api:
        condition: service_healthy
    restart: unless-stopped

  # API layer: validates requests, enqueues tasks
  api:
    build: ./api
    environment:
      - BROKER_URL=redis://broker:6379/0
      - RESULT_BACKEND=redis://broker:6379/1
      - AUTH_JWT_SECRET=${JWT_SECRET}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 15s
      timeout: 5s
      retries: 3
    depends_on:
      broker:
        condition: service_healthy
    restart: unless-stopped

  # Agent workers: the core reasoning engines
  agent-worker:
    build: ./worker
    environment:
      - BROKER_URL=redis://broker:6379/0
      - RESULT_BACKEND=redis://broker:6379/1
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - TOOL_REGISTRY_URL=http://tool-registry:8080
      - MEMORY_STORE_URL=postgresql://agent:${DB_PASSWORD}@memory-db/agent_memory
    healthcheck:
      test: ["CMD-SHELL", "celery -A tasks inspect ping -d agent-worker@$$HOSTNAME"]
      interval: 30s
      timeout: 10s
      retries: 3
    depends_on:
      broker:
        condition: service_healthy
      memory-db:
        condition: service_healthy
      tool-registry:
        condition: service_healthy
    restart: unless-stopped
    deploy:
      replicas: 4
      resources:
        limits:
          cpus: '2'
          memory: 4G
        reservations:
          cpus: '1'
          memory: 2G

  # Message broker cluster
  broker:
    image: redis:7.2.4-alpine
    command: redis-server --appendonly yes --maxmemory 2gb --maxmemory-policy allkeys-lru
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - broker-data:/data
    restart: unless-stopped

  # Persistent memory store with vector support
  memory-db:
    image: pgvector/pgvector:pg16-0.7.0
    environment:
      POSTGRES_USER: agent
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: agent_memory
    volumes:
      - memory-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U agent -d agent_memory"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  # Tool registry: dynamic tool discovery
  tool-registry:
    build: ./tool-registry
    volumes:
      - ./tool-defs:/definitions:ro
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
    restart: unless-stopped

  # Tool: Web search
  search-tool:
    build: ./tools/search
    environment:
      - BRAVE_API_KEY=${BRAVE_API_KEY}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
      interval: 30s
    restart: unless-stopped

  # Tool: Code sandbox (isolated network)
  code-sandbox:
    build: ./tools/sandbox
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8082/health"]
      interval: 30s
    networks:
      - app-network
      - sandbox-isolated
    restart: unless-stopped

  # Observability: metrics collection
  prometheus:
    image: prom/prometheus:v2.51.0
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus-data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
    ports:
      - "9090:9090"
    restart: unless-stopped

  # Observability: dashboards
  grafana:
    image: grafana/grafana:10.4.0
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
    volumes:
      - grafana-data:/var/lib/grafana
      - ./grafana-dashboards:/etc/grafana/provisioning/dashboards:ro
    restart: unless-stopped

volumes:
  broker-data:
  memory-data:
  prometheus-data:
  grafana-data:

networks:
  app-network:
  sandbox-isolated:
    driver: bridge
    internal: true

This stack represents a battle-tested production deployment. The gateway handles TLS termination and basic auth, the API layer validates and enqueues tasks, the agent workers process tasks asynchronously with proper health checks, and the observability stack provides visibility into every component. The code sandbox runs on an isolated internal network, preventing any executed code from making unauthorized outbound connections.

Deployment Workflow

The production deployment workflow with Docker Compose typically follows this sequence:

# 1. Validate the Compose file syntax
docker compose -f docker-compose.prod.yml config > /dev/null && echo "Config valid"

# 2. Build all images with production tags
docker compose -f docker-compose.prod.yml build --no-cache

# 3. Start the stack in detached mode
docker compose -f docker-compose.prod.yml up -d

# 4. Verify all health checks pass
docker compose -f docker-compose.prod.yml ps
# Look for all services showing "healthy" status

# 5. Check logs for any startup errors
docker compose -f docker-compose.prod.yml logs --tail=50 agent-worker

# 6. Scale workers if needed
docker compose -f docker-compose.prod.yml up -d --scale agent-worker=8

# 7. Graceful restart after config changes
docker compose -f docker-compose.prod.yml up -d --no-deps --build agent-worker

Common Pitfalls and How to Avoid Them

• Pitfall: Agents starting before dependencies are ready. Always use depends_on with condition: service_healthy (not just service_started) to ensure databases and brokers are truly accepting connections.
• Pitfall: Unbounded memory growth. Agent containers running LLM chains can leak memory over time. Set Docker memory limits and implement periodic worker recycling (Celery's worker_max_tasks_per_child is excellent for this).
• Pitfall: Lost tasks on restart. Use broker configurations with persistence enabled (Redis AOF, RabbitMQ durable queues) so queued tasks survive broker restarts.
• Pitfall: Hardcoded service locations. Use Compose service names as DNS hostnames (e.g., broker:6379) rather than IP addresses. Compose handles DNS resolution automatically.
• Pitfall: Debugging without observability. Always include at minimum a log aggregator and the Flower dashboard (for Celery) or equivalent. Agents are complex state machines—you need visibility when things go wrong.

Conclusion

Deploying agents with Docker Compose using these production patterns transforms fragile, single-process prototypes into robust, observable, and maintainable systems. The key insight is that agent deployments are not just about running LLM code—they're about orchestrating a constellation of services that together create reliable autonomous behavior. By containerizing each component with clear interfaces, health checks, resource limits, and persistence, you build a foundation that can grow from a handful of agent tasks per minute to thousands, while remaining debuggable and recoverable when the inevitable LLM failures occur. Start with Pattern 1 for simple use cases, adopt Pattern 2 when throughput matters, and layer in sidecars and full observability as your agent system matures into a critical production dependency.