Docker Stack Deployments: Best Practices and Common Pitfalls

What is Docker Stack Deploy?

Docker Stack Deploy is the primary command used to deploy and manage a group of interrelated services as a single unit — called a stack — into a Docker Swarm cluster. It relies on a Compose file (version 3.x or higher) that describes services, networks, volumes, secrets, and configs. When you run docker stack deploy, Docker reads the Compose file, creates or updates the requested services, and hands over their lifecycle to the Swarm orchestration engine. This engine schedules containers across available nodes, monitors their health, handles rolling updates, and restores the declared state if anything drifts.

Think of a stack as the Swarm-mode equivalent of docker-compose up but with cluster‑wide scheduling, built‑in service discovery, secret management, and declarative rollouts. Stacks are the recommended way to run multi‑service applications in production‑grade Docker environments.

Why It Matters

Running containers in production demands more than just starting processes. You need resilience, scalability, zero‑downtime deployments, secure configuration distribution, and the ability to roll back when something goes wrong. Docker stacks give you all of that out of the box:

• Declarative configuration – define the desired state once, and the system converges to it.
• Service discovery & load balancing – internal DNS and an ingress mesh automatically route traffic to service replicas.
• Rolling updates – replace containers gradually with fine‑grained control (parallelism, delay, failure thresholds).
• Secrets & configs – mount sensitive and non‑sensitive data as files inside containers without baking them into images.
• Resource limits & constraints – guarantee performance isolation and control where workloads land.
• Multi‑node coordination – a single command deploys an entire application across dozens of Swarm nodes.

Ignoring stacks and sticking with standalone docker run or even docker-compose on a single host quickly becomes a bottleneck when you need high availability and smooth change management.

How to Use Docker Stack Deploy

Prerequisites

Before you deploy your first stack, you need an active Swarm cluster. Initialize one on a manager node:

docker swarm init --advertise-addr <MANAGER-IP>

Add worker nodes with docker swarm join as needed. Verify the cluster is healthy with docker node ls. All stack commands must be issued from a Swarm manager.

Compose File Structure for Stacks

A stack‑ready Compose file uses version 3.x (preferably 3.8 or later for full Swarm support). The critical difference from a regular docker-compose.yml is the deploy key placed under each service. It controls scheduling, updates, restart policy, resource limits, and placement constraints.

Below is a minimal but complete example:

version: '3.8'

services:
  web:
    image: nginx:stable-alpine
    ports:
      - "80:80"
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s
        order: start-first
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
    networks:
      - frontend

  api:
    image: myregistry.example.com/my-api:2.1
    environment:
      DB_HOST: db
    deploy:
      replicas: 2
      resources:
        limits:
          cpus: '0.50'
          memory: 256M
    networks:
      - frontend
      - backend

  db:
    image: postgres:15-alpine
    secrets:
      - pg_password
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/pg_password
    volumes:
      - db-data:/var/lib/postgresql/data
    deploy:
      placement:
        constraints:
          - node.role == worker
          - node.labels.storage == ssd
    networks:
      - backend

networks:
  frontend:
    driver: overlay
  backend:
    driver: overlay

volumes:
  db-data:

secrets:
  pg_password:
    external: true

Deploying the Stack

With the Compose file ready, run:

docker stack deploy -c docker-compose.yml myapp

Docker will create the networks, volumes (if they don’t exist), secrets (if declared as external they must already exist), and then the services. You can verify everything with:

docker stack ls
docker stack services myapp
docker stack ps myapp

Updating a Stack

To apply changes (image tag bump, replica count, environment variable, etc.), simply modify the Compose file and run the same docker stack deploy command again. Docker compares the new desired state with the current one and issues service updates. The rolling update parameters defined in deploy.update_config control the pace and failure behavior.

Removing a Stack

Tear down the entire application with:

docker stack rm myapp

This removes all services, but networks and volumes defined in the Compose file survive unless you manually delete them. Secrets that are externally created are not removed by stack deletion — you must delete them separately.

Best Practices

1. Use the deploy Section Extensively

Never omit deploy. It is your control panel for production resilience. At a minimum, set replicas, restart_policy, and update_config.

deploy:
  replicas: 2
  restart_policy:
    condition: on-failure
  update_config:
    parallelism: 1
    delay: 10s
    failure_action: rollback
    max_failure_ratio: 0.3

2. Define Resource Limits for Every Service

Without limits, a buggy container can consume all CPU or memory on a node, starving neighbours. Always set both limits and, where possible, reservations.

deploy:
  resources:
    limits:
      cpus: '0.75'
      memory: 512M
    reservations:
      cpus: '0.25'
      memory: 128M

3. Prefer Secrets for Sensitive Data

Never pass passwords or API keys via environment variables baked into images or Compose files committed to version control. Create secrets externally and reference them as external: true:

echo "supersecret" | docker secret create pg_password -

Then mount them in services:

secrets:
  - pg_password

Inside the container, the secret appears as a file in /run/secrets/pg_password. Many official images support a _FILE variant environment variable to read from a file.

4. Use Configs for Non‑Sensitive Configuration Files

For things like nginx config templates, JSON settings, or feature flags, use Docker configs instead of volumes. They are mounted as read‑only files and can be versioned independently of the image.

configs:
  nginx_conf:
    file: ./nginx.conf
services:
  web:
    configs:
      - source: nginx_conf
        target: /etc/nginx/conf.d/default.conf

5. Design Overlay Networks Carefully

All inter‑service communication should happen over an overlay network. Create dedicated networks per tier (frontend, backend) to isolate traffic. Avoid relying on the default ingress network for service‑to‑service calls; it is meant for external traffic only.

networks:
  frontend:
    driver: overlay
  backend:
    driver: overlay
    internal: true   # prevents external access

6. Use Placement Constraints and Preferences

Guide the scheduler to run workloads on appropriate hardware or to spread replicas across racks/availability zones.

deploy:
  placement:
    constraints:
      - node.labels.region == us-east
    preferences:
      - spread: node.labels.rack

7. Implement Health Checks

Swarm relies on container health status for rolling updates and service availability. Define a healthcheck inside your Dockerfile or directly in the Compose file:

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost/health"]
  interval: 30s
  timeout: 5s
  retries: 3
  start_period: 10s

8. Always Use --with-registry-auth for Private Registries

When your images live in a private registry (e.g., AWS ECR, Harbor), the Swarm nodes need credentials. Pass them by logging in on the manager and adding the flag:

docker stack deploy --with-registry-auth -c docker-compose.yml myapp

This distributes the manager’s registry credentials to all nodes for the lifetime of the stack deployment. Without it, workers will fail to pull images with authentication errors.

9. Version Your Stacks and Compose Files

Treat your Compose file like infrastructure as code. Store it in a Git repository, tag releases, and use CI/CD pipelines to run docker stack deploy. Avoid ad‑hoc edits directly on the manager node.

10. Use Environment Variables Judiciously

Non‑secret configuration (debug flags, upstream URLs) can come from environment variables. Set them via the Compose environment block or pass a file with env_file. But remember that environment variables are visible in docker service inspect, so keep secrets out.

Common Pitfalls

Pitfall 1: Deploying Without Swarm Mode Enabled

Running docker stack deploy on a standalone Docker engine (not part of a Swarm) will fail with a message like “this node is not a swarm manager”. Always ensure you have run docker swarm init or docker swarm join as appropriate. A quick check: docker info | grep Swarm should show Swarm: active.

Pitfall 2: Wrong Compose File Version

Stacks ignore version 2.x Compose files. You must use version 3.0 or higher. Attempting to deploy a version 2 file will result in an error or unexpected behavior because features like deploy and secrets are not recognized.

Pitfall 3: Ignoring Placement Constraints Leading to Pending Services

If you set a placement constraint like node.labels.env == production but no node has that label, the service will remain in “pending” state forever. Use docker node update --label-add env=production <node-id> to add labels before deploying.

Pitfall 4: Using build Instead of Pre‑built Images

Stack deploy does not support build: directives. Swarm nodes must pull existing images from a registry. Always build, tag, and push images before deploying a stack. If you accidentally leave a build: section, the deploy command will ignore it (or fail depending on version), and the service will never start because no image is available.

Pitfall 5: Forgetting to Push Images or Using latest Tag

Using the latest tag without a specific digest means each node may cache a different version depending on when it pulled. Always use explicit version tags (e.g., myapp:2.1.4) and ensure the image is pushed to the registry before deploying.

Pitfall 6: Relying on depends_on for Startup Order

Docker Compose’s depends_on only guarantees order during initial startup on a single host. In Swarm stacks, services start independently across multiple nodes. There is no built‑in mechanism to wait for a database to be ready before starting the API. Implement retry logic or use an init container pattern (e.g., wait-for-it script in an entrypoint).

Pitfall 7: Using container_name or links

These Compose directives are ignored by stacks. Service discovery happens via DNS using the service name, not container names. Remove them from stack‑oriented Compose files to avoid confusion.

Pitfall 8: Exposing Ports Without Ingress or Host Mode Awareness

When you publish a port in a Swarm service, the default mode is ingress (routing mesh), which distributes incoming traffic across all replicas regardless of the node they run on. If you need to bind only on a specific node (e.g., for a stateful set), use mode: host and placement constraints:

ports:
  - published: 8080
    target: 80
    mode: host
deploy:
  placement:
    constraints:
      - node.hostname == specific-node

Misunderstanding this leads to traffic going to unintended replicas.

Pitfall 9: Unintended Secret Exposure via docker inspect

Even though secrets are mounted as files, the service definition in docker service inspect shows which secrets are assigned. It does not show secret values, but if you accidentally log environment variables containing secrets, those become visible. Always use _FILE variants or read from /run/secrets.

Pitfall 10: Missing --with-registry-auth for Private Registries

As mentioned earlier, omitting this flag leads to “image pull failed” errors on worker nodes. Even if the manager can pull the image, workers will not have credentials unless you explicitly pass them.

Pitfall 11: Deleting a Stack Without Cleaning Up Volumes or Secrets

docker stack rm removes services and networks defined in the Compose file, but volumes and externally created secrets persist. Over time, dangling volumes consume disk space, and old secrets clutter the secret store. Implement a cleanup routine: docker volume prune and docker secret rm when appropriate.

Pitfall 12: Overlapping Stack Names or Ports

Two stacks can share the same overlay network, but if they both try to publish the same host port, the second deploy will fail. Use unique published ports or rely on ingress routing with different stack‑specific virtual IPs (no port clash if services use internal overlay networks only). Plan port ranges carefully.

Pitfall 13: Not Testing Rolling Update Parameters

Incorrect update_config can cause a bad update to roll through completely before failing, or a rollback can get stuck. For example, setting parallelism: 0 is invalid. Always test in a staging Swarm: simulate a failing health check and verify the failure_action: rollback works as expected.

Pitfall 14: Confusing docker-compose and docker stack Commands

Remember: docker-compose targets a single host and uses Compose file version 2 or 3 (with some swarm‑only features ignored). docker stack requires Swarm and version 3.x. Mixing them leads to frustration — for example, docker-compose up -d on a Swarm manager starts standalone containers, not Swarm services, losing all scheduling and rolling update features.

Conclusion

Docker Stack Deploy is a powerful abstraction that turns a multi‑service Compose file into a resilient, scalable application running on a Swarm cluster. By understanding its Swarm‑mode foundations — declarative state, overlay networking, secrets, and rolling updates — you can avoid the common traps that plague production deployments. Start by enforcing resource limits, externalizing configuration, and pinning image versions. Design your Compose files with the deploy section at their core. Test updates and rollbacks in a non‑production Swarm, and always use --with-registry-auth for private registries. When you follow these best practices, stacks become a reliable, repeatable, and safe mechanism to deliver applications from development all the way to a highly available production environment.