← Back to DevBytes

Docker Health Checks: Best Practices and Common Pitfalls

What is a Docker Health Check?

A Docker health check is a mechanism that allows you to define a command that Docker will periodically run inside a running container to verify whether the container is still functioning correctly. Think of it as an automated heartbeat monitor for your containers — it tells Docker (and orchestration platforms) whether the application inside the container is actually working, not just whether the container process is running.

By default, Docker only monitors whether the container's main process (PID 1) is alive. If the process crashes, the container exits. But what if your web server is running but stuck in a deadlock, unable to serve requests? What if your database process is up but refusing connections? The container appears "running" from Docker's perspective, yet the application is effectively dead. Health checks bridge this gap.

The HEALTHCHECK Instruction in Dockerfile

The HEALTHCHECK instruction tells Docker how to test a container to check that it is still working. Here's the basic syntax:

HEALTHCHECK [OPTIONS] CMD <command> <arg1> <arg2> ...

The command runs inside the container and its exit status determines the health:

The available options control the timing and retry behavior:

A Simple Example: Health Check for a Web Server

FROM nginx:alpine

# Install curl for the health check
RUN apt-get update && apt-get install -y curl || apk add --no-cache curl

# Copy your application files
COPY ./html /usr/share/nginx/html

# Define health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD curl -f http://localhost/ || exit 1

In this example, Docker will:

The -f flag on curl makes it return a non-zero exit code for HTTP errors (4xx, 5xx), which correctly triggers a failure.

How to Inspect Health Status

You can view the health status of running containers using standard Docker commands:

# List containers with health status
docker ps

# Output includes a STATUS column showing health
# Example: Up 2 hours (healthy)

# Detailed health information
docker inspect --format='{{json .State.Health}}' container_name | jq .

# Example output:
# {
#   "Status": "healthy",
#   "FailingStreak": 0,
#   "Log": [
#     {
#       "Start": "2024-01-15T10:30:00Z",
#       "End": "2024-01-15T10:30:01Z",
#       "ExitCode": 0,
#       "Output": "..."
#     }
#   ]
# }

The health status appears in docker ps as one of: (healthy), (unhealthy), or (starting) (during the start period or before the first check completes).

Health Checks in Docker Compose

When using Docker Compose, you can define health checks directly in the docker-compose.yml file. This approach is often cleaner than embedding everything in a Dockerfile, and it allows environment-specific adjustments.

version: '3.8'

services:
  web:
    image: my-web-app:latest
    ports:
      - "8080:80"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 20s

  database:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 15s

Note the two command formats in Compose:

Always prefer CMD (exec form) when possible, as it avoids spawning an extra shell process. Use CMD-SHELL only when you need shell features like |, &&, or environment variable expansion.

Health Checks as Service Dependencies

One of the most powerful features of health checks is using them to control startup order in Docker Compose with depends_on:

version: '3.8'

services:
  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s

  app:
    image: my-app:latest
    depends_on:
      db:
        condition: service_healthy
    ports:
      - "3000:3000"

With condition: service_healthy, Compose will not start the app service until the db health check passes. This is vastly superior to blind sleep-based waiting. Note that condition was removed in Compose v3 but restored in newer versions of Docker Compose (v2.20+). If you're using an older version, you can use condition: service_started as a fallback and handle retry logic in your application.


Why Health Checks Matter

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Health checks aren't just a nice-to-have — they fundamentally improve container reliability in several critical ways:

1. Orchestration Integration

Orchestrators like Docker Swarm, Kubernetes (via the Docker runtime), and AWS ECS use health check status to make automated decisions. Unhealthy containers get restarted or replaced automatically. Without health checks, an orchestrator might keep a non-functional container running indefinitely simply because the process hasn't exited.

2. Zero-Downtime Deployments

During rolling updates, new containers are only considered ready when they pass their health check. If a new version fails its health check, the orchestrator can halt the deployment and roll back, preventing a bad release from reaching production.

3. Load Balancer Integration

Reverse proxies and load balancers (Traefik, HAProxy, NGINX) can query container health to automatically remove unhealthy backends from the pool, preventing traffic from reaching broken instances.

4. Startup Ordering

As shown above, health checks enable proper dependency-based startup sequences, eliminating race conditions between services.

5. Operational Visibility

Health status provides immediate insight into whether your services are truly operational, not just whether the container exists. This is invaluable in debugging and monitoring scenarios.


Best Practices for Docker Health Checks

1. Create a Dedicated Health Check Endpoint

For web applications, don't simply check the homepage. Create a dedicated /health or /healthz endpoint that performs internal integrity checks:

# In your application - example using Python FastAPI
@app.get("/health")
async def health_check():
    # Verify database connectivity
    try:
        await db.execute("SELECT 1")
    except Exception:
        return {"status": "unhealthy", "reason": "database_unreachable"}, 503
    
    # Verify cache connectivity
    try:
        await cache.ping()
    except Exception:
        return {"status": "unhealthy", "reason": "cache_unreachable"}, 503
    
    return {"status": "healthy"}, 200

Your Dockerfile health check then becomes trivially simple:

HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

This endpoint should be lightweight — avoid expensive operations. It should not perform deep validation that could fail under normal load. The goal is to verify connectivity and basic functionality, not to run a full integration test suite.

2. Use Application-Native Tools, Not Shell Scripts

Whenever possible, use tools that are purpose-built for health checking rather than cobbling together shell commands:

These tools properly handle authentication, timeouts, and edge cases that a naive nc (netcat) port check would miss.

3. Tune the Timing Parameters Realistically

The default values (30s interval, 30s timeout, 0s start-period, 3 retries) are often inappropriate for production workloads. Here's how to think about tuning them:

4. Keep Health Check Commands Lightweight and Side-Effect-Free

A health check should be:

Avoid health checks that:

5. Prefer the Exec Form Over the Shell Form

Use the exec form (JSON array) of the command to avoid spawning an extra shell process:

# Good - exec form, no extra shell
HEALTHCHECK --interval=30s --timeout=5s \
  CMD ["curl", "-f", "http://localhost/health"]

# Avoid - shell form, spawns /bin/sh for every check
HEALTHCHECK --interval=30s --timeout=5s \
  CMD curl -f http://localhost/health

In Compose, the equivalent is:

# Good
test: ["CMD", "curl", "-f", "http://localhost/health"]

# Only when shell features are needed
test: ["CMD-SHELL", "pg_isready -U $POSTGRES_USER"]

The exec form is more efficient, avoids signal-handling issues, and prevents subtle bugs with shell variable expansion.

6. Include the Health Check Tool in the Production Image

Don't rely on tools that aren't in your final image. If your health check uses curl, ensure curl is installed in the production stage of your multi-stage build:

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --production

FROM node:20-alpine AS runner
RUN apk add --no-cache curl   # Install curl for health checks
COPY --from=builder /app /app
WORKDIR /app
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1
CMD ["node", "server.js"]

If you absolutely cannot include the tool in the final image, consider writing a tiny health check in the same language as your application (e.g., a small Go or Node.js script) that uses only built-in libraries.

7. Use Different Health Checks for Different Environments

What constitutes "healthy" may differ between development and production. In development, you might want a faster interval and shorter start period. Use Compose override files or environment variables to adjust:

# docker-compose.yml (base)
services:
  app:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: ${HEALTH_INTERVAL:-30s}
      start_period: ${HEALTH_START_PERIOD:-30s}
      retries: ${HEALTH_RETRIES:-3}

# docker-compose.dev.yml (override)
services:
  app:
    healthcheck:
      interval: 5s
      start_period: 5s

8. Log Health Check Results Meaningfully

The health check command's stdout and stderr are captured in the container's health log, visible via docker inspect. Make your health check output useful for debugging:

HEALTHCHECK --interval=30s --timeout=5s \
  CMD curl -f -v http://localhost/health 2>&1 || exit 1

The -v flag adds verbose output that can help diagnose why a check is failing.


Common Pitfalls (and How to Avoid Them)

Pitfall 1: The Infinite Start Period Trap

The problem: Setting an excessively long --start-period masks genuine startup failures. If your application crashes during the start period, the container stays in (starting) status until the period ends, delaying detection.

The fix: Set the start period to the real maximum startup time of your application, not an arbitrary large number. If your app typically starts in 10 seconds, setting start-period to 300 seconds means a crash at 15 seconds goes unnoticed for nearly 5 minutes.

# Bad - hides startup failures for 5 minutes
HEALTHCHECK --start-period=300s --interval=30s CMD curl -f http://localhost/ || exit 1

# Good - realistic startup window
HEALTHCHECK --start-period=30s --interval=30s CMD curl -f http://localhost/ || exit 1

Pitfall 2: Port Checking Without Protocol Verification

The problem: Many tutorials suggest using nc (netcat) to check if a port is open. This only verifies that something is listening on the port — not that it's the correct application or that it's functioning properly.

# Bad - only checks if port 80 is open, doesn't verify HTTP response
HEALTHCHECK CMD nc -z localhost 80 || exit 1

A port can be open while the application behind it is completely broken. Always verify application-level behavior:

# Good - verifies actual HTTP response
HEALTHCHECK CMD curl -f http://localhost/ || exit 1

Pitfall 3: Health Check That Depends on External Services

The problem: A health check that calls an external API or checks an external dependency can cause cascading failures. If your health check endpoint internally verifies connectivity to an upstream service that is temporarily down, your container gets marked unhealthy even though your application code is fine.

The fix: Design your health endpoint to report on internal health only — is the process running, can it respond to requests, are its internal queues healthy? Use a separate /ready (readiness) endpoint for startup-time dependency checks that block traffic routing, and keep /health for ongoing liveness checks that don't depend on external services.

# /health - liveness: is the process alive?
# Returns 200 as long as the process can respond
@app.get("/health")
def health():
    return {"status": "alive"}

# /ready - readiness: can we serve traffic?
# Checks dependencies, used for startup and load balancer routing
@app.get("/ready")
def ready():
    if not database_connected():
        return {"status": "not_ready"}, 503
    return {"status": "ready"}

Use /health for the Docker health check (liveness) and /ready for load balancer health checks (readiness).

Pitfall 4: Health Check Timeout Equal to or Greater Than Interval

The problem: If the health check timeout is longer than the interval, you can end up with multiple overlapping health check processes running simultaneously. Each check consumes resources, and if the application is slow (not dead, just slow), the overlapping checks compound the problem.

# Dangerous - timeout (60s) > interval (30s)
HEALTHCHECK --interval=30s --timeout=60s CMD curl -f http://localhost/ || exit 1

The fix: Always ensure timeout < interval. A good rule of thumb is timeout should be at most half of interval:

# Safe - timeout (10s) < interval (30s)
HEALTHCHECK --interval=30s --timeout=10s CMD curl -f http://localhost/ || exit 1

Pitfall 5: Using curl Without -f Flag

The problem: curl returns exit code 0 even for HTTP 4xx and 5xx responses. Without -f (fail), a health check against a broken application returning 500 Internal Server Error will still report success.

# Bad - curl succeeds on HTTP 500
HEALTHCHECK CMD curl http://localhost/ || exit 1

The fix: Always use curl -f or curl --fail:

# Good - curl fails on any HTTP error
HEALTHCHECK CMD curl -f http://localhost/ || exit 1

Alternatively, check for a specific expected status code:

# Checks for exactly 200
HEALTHCHECK CMD curl -s -o /dev/null -w "%{http_code}" http://localhost/health | grep -q 200 || exit 1

Pitfall 6: Health Check With No Retries

The problem: A single transient failure immediately marks the container unhealthy. Network blips, momentary CPU spikes, or GC pauses in the JVM can cause occasional health check timeouts that are not indicative of real problems.

The fix: Always use --retries with at least 2 or 3:

# Fragile - any single failure marks unhealthy
HEALTHCHECK --interval=30s --timeout=5s CMD curl -f http://localhost/ || exit 1

# Robust - requires 3 consecutive failures
HEALTHCHECK --interval=30s --timeout=5s --retries=3 CMD curl -f http://localhost/ || exit 1

Pitfall 7: Forgetting That Health Checks Run Forever

The problem: The health check command continues to run periodically for the entire lifetime of the container. A command that allocates memory without freeing it, or that opens connections without closing them, will slowly leak resources.

The fix: Ensure your health check command cleans up after itself. If using curl, it automatically closes connections. If writing a custom script, ensure it exits cleanly and doesn't leave open file descriptors or connections.

Pitfall 8: Using Shell Form and Accidentally Breaking Exit Code Propagation

The problem: When using CMD-SHELL or the shell form of HEALTHCHECK, the exit code of the last command in the chain is used. If you chain commands with &&, an early failure correctly propagates. But if you use pipes or command substitution, the exit code might be from the wrong command:

# Bad - exit code is from grep, but curl failure is masked if grep succeeds on empty output
HEALTHCHECK CMD curl http://localhost/health | grep -q "healthy" || exit 1

In bash, the exit code of a pipeline is the exit code of the last command unless pipefail is set:

# Better - pipefail ensures any command failure propagates
HEALTHCHECK CMD-SHELL set -o pipefail && curl -f http://localhost/health | grep -q "healthy" || exit 1

Or simply avoid complex shell pipelines in health checks.

Pitfall 9: Not Testing Health Checks During Development

The problem: Developers often write health checks, see the container status flip to (healthy), and assume everything works. But they never test what happens when the application is actually broken.

The fix: Deliberately break your application and verify that the health check correctly detects the failure:

# Test unhealthy detection
# 1. Start the container normally
docker run -d --name test-app my-app:latest

# 2. Wait for healthy status
docker ps --filter "name=test-app"

# 3. Simulate failure (e.g., kill the app process, stop the database, fill the disk)
docker exec test-app pkill -STOP node   # Freeze the process without exiting

# 4. Verify the health check eventually reports unhealthy
watch docker ps --filter "name=test-app"
# Should transition from (healthy) -> (unhealthy) after interval * retries

Pitfall 10: Ignoring Health Check in Orchestrated Environments

The problem: In Kubernetes, Docker health checks are not directly used. Kubernetes has its own liveness and readiness probe system. However, if you're running Docker containers directly (without Kubernetes), the Docker health check is your primary automated health verification.

If you're migrating from Docker to Kubernetes, translate your Docker health check into Kubernetes probes:

# Kubernetes equivalent of Docker HEALTHCHECK
# livenessProbe acts like Docker health check
livenessProbe:
  httpGet:
    path: /health
    port: 3000
  initialDelaySeconds: 10    # Equivalent to --start-period
  periodSeconds: 30          # Equivalent to --interval
  timeoutSeconds: 5          # Equivalent to --timeout
  failureThreshold: 3        # Equivalent to --retries

Complete Real-World Example

Let's put everything together with a production-ready multi-service application:

# docker-compose.yml
version: '3.8'

services:
  # PostgreSQL with proper health check
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: securepassword
      POSTGRES_DB: appdb
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 15s
    restart: unless-stopped

  # Redis with native ping check
  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 5s
    restart: unless-stopped

  # Node.js application with dedicated /health endpoint
  app:
    build:
      context: ./app
      dockerfile: Dockerfile
    environment:
      DATABASE_URL: postgresql://appuser:securepassword@postgres:5432/appdb
      REDIS_URL: redis://redis:6379
    ports:
      - "3000:3000"
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s
    restart: unless-stopped

volumes:
  pgdata:

And here's the corresponding Dockerfile for the application:

# Dockerfile
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --production

FROM node:20-alpine AS runner
RUN apk add --no-cache curl
COPY --from=builder /app /app
WORKDIR /app
COPY server.js health.js ./
EXPOSE 3000

HEALTHCHECK \
  --interval=30s \
  --timeout=5s \
  --start-period=20s \
  --retries=3 \
  CMD ["curl", "-f", "http://localhost:3000/health"]

CMD ["node", "server.js"]

And the application health endpoint:

// health.js - lightweight health check endpoint
const express = require('express');
const router = express.Router();

router.get('/health', async (req, res) => {
  // Quick, lightweight checks only
  const health = {
    status: 'healthy',
    timestamp: new Date().toISOString(),
    uptime: process.uptime(),
    memory: process.memoryUsage().rss
  };
  
  // Simple DB connectivity check with short timeout
  try {
    const dbCheck = await Promise.race([
      db.query('SELECT 1'),
      new Promise((_, reject) => 
        setTimeout(() => reject(new Error('db_timeout')), 3000)
      )
    ]);
    health.database = 'connected';
  } catch (err) {
    health.status = 'unhealthy';
    health.database = 'disconnected';
    return res.status(503).json(health);
  }

  // Simple Redis check
  try {
    const redisCheck = await Promise.race([
      redis.ping(),
      new Promise((_, reject) => 
        setTimeout(() => reject(new Error('redis_timeout')), 2000)
      )
    ]);
    health.redis = 'connected';
  } catch (err) {
    health.status = 'unhealthy';
    health.redis = 'disconnected';
    return res.status(503).json(health);
  }

  res.status(200).json(health);
});

module.exports = router;

This example demonstrates several best practices: dedicated health endpoints, native tooling for databases (pg_isready, redis-cli ping), proper timing parameters, dependency ordering via depends_on with service_healthy, and lightweight checks with timeouts to prevent cascading stalls.


Conclusion

Docker health checks are a deceptively simple feature with profound implications for container reliability. A well-designed health check transforms your containers from opaque black boxes into observable, self-reporting units that orchestration systems can manage intelligently. The key takeaways are: create dedicated, lightweight health endpoints; use application-native checking tools rather than generic port checks; tune your timing parameters based on real application behavior rather than relying on defaults; keep health checks side-effect-free and independent of external services; and always test failure scenarios to ensure your health check actually detects problems when they occur. By avoiding the common pitfalls outlined above and following the best practices, you'll build containerized applications that are resilient, self-healing, and production-ready.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles