Troubleshooting Varnish Cache: Common Issues and Fixes

Introduction to Varnish Cache Troubleshooting

Varnish Cache is a high-performance HTTP accelerator designed for content-heavy dynamic web applications. Sitting in front of your web server, it caches responses to reduce load and dramatically improve page load times. However, when Varnish misbehaves, it can serve stale content, bypass caching entirely, or break authentication flows. This tutorial walks you through the most common Varnish issues and provides practical fixes you can apply immediately.

Why Troubleshooting Varnish Matters

A misconfigured Varnish instance can be worse than no caching at all. It can serve outdated content to users, leak sensitive data across sessions, or silently fail to cache anything—leaving you with the overhead of Varnish but none of the benefits. Understanding how to diagnose and fix these issues is essential for any developer or sysadmin operating Varnish in production.

Issue 1: Varnish Not Caching Anything

The most frequent complaint is that Varnish appears to be running but every request results in a cache miss. This usually stems from response headers, request methods, or VCL logic that prevents caching.

Diagnosing with varnishlog

Use varnishlog to inspect what Varnish is actually doing with each request. Look for the VCL_return tag to see why a response was not cached.

varnishlog -g request -q 'RespStatus == 200' | grep -E 'VCL_return|TTL|Fetch'

Common reasons Varnish refuses to cache include:

• The backend sends a Set-Cookie header on every response
• The response includes Cache-Control: no-cache or private
• The request method is not GET or HEAD
• The response status code is not in the cacheable range
• The Authorization header is present in the request

Fixing the Set-Cookie Problem

Many backends (especially WordPress and Magento) set a session cookie on every page load. Varnish will not cache responses with Set-Cookie by default. You can strip unnecessary cookies in your VCL file.

sub backend_response {
    # Remove Set-Cookie for static assets
    if (bereq.url ~ "\.(css|js|png|jpg|jpeg|gif|ico|svg|woff2?)$") {
        unset beresp.http.Set-Cookie;
    }

    # Remove Set-Cookie for anonymous page views
    if (bereq.url ~ "^/$" || bereq.url ~ "^/(about|contact|blog)") {
        unset beresp.http.Set-Cookie;
    }

    set beresp.ttl = 5m;
    return (deliver);
}

Overriding Cache-Control Headers

If your backend sends restrictive cache headers but you know the content is safe to cache, you can override them in vcl_backend_response.

sub backend_response {
    if (bereq.url ~ "\.(css|js|png|jpg|jpeg|gif|ico|svg|woff2?)$") {
        unset beresp.http.Cache-Control;
        unset beresp.http.Set-Cookie;
        set beresp.http.Cache-Control = "public, max-age=86400";
        set beresp.ttl = 24h;
        set beresp.uncacheable = false;
        return (deliver);
    }
}

Issue 2: Serving Stale Content

The opposite problem is serving content that never seems to update. Users see old prices, outdated articles, or removed pages. This happens when cache TTLs are too long or cache invalidation is not configured.

Setting Appropriate TTLs

Balance freshness with performance by setting reasonable TTLs based on content type.

sub backend_response {
    if (bereq.url ~ "\.(css|js|png|jpg|jpeg|gif|ico|svg|woff2?)$") {
        set beresp.ttl = 24h;
    } else if (bereq.url ~ "/api/") {
        set beresp.ttl = 30s;
    } else {
        set beresp.ttl = 2m;
    }

    # Enable grace mode for stale-while-revalidate behavior
    set beresp.grace = 1h;
    return (deliver);
}

Implementing Cache Purging

For immediate content updates, implement cache purging. First, configure an ACL for purge requests.

acl purge {
    "localhost";
    "127.0.0.1";
    "::1";
    "192.168.1.0"/24;
}

sub vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge) {
            return (synth(403, "Forbidden"));
        }
        return (purge);
    }

    # Support BAN for pattern-based purging
    if (req.method == "BAN") {
        if (!client.ip ~ purge) {
            return (synth(403, "Forbidden"));
        }
        ban("req.url ~ " + req.url);
        return (synth(200, "Ban added"));
    }
}

To purge a specific URL from the command line:

curl -X PURGE http://localhost:6081/my-page

To ban all cached objects matching a pattern:

curl -X BAN http://localhost:6081/category/news

Issue 3: High Hit-For-Miss Rate

A Hit-For-Miss occurs when Varnish caches the fact that an object is not cacheable, preventing repeated backend requests. While this protects your backend, an unexpectedly high rate suggests misconfiguration.

Identifying Hit-For-Miss Objects

Check your hit-for-miss statistics with varnishstat:

varnishstat -1 | grep n_objectcore

Use varnishlog to find which URLs are being marked as uncacheable:

varnishlog -g request -q 'VCL_return eq "hit_for_miss"'

Fixing Unintentional Hit-For-Miss

Review your VCL for places where you set beresp.uncacheable = true or return hit_for_miss. Ensure you only mark genuinely uncacheable content this way.

sub backend_response {
    # Only mark as hit-for-miss for truly uncacheable responses
    if (bereq.url ~ "/api/user/" || bereq.url ~ "/checkout/") {
        set beresp.uncacheable = true;
        set beresp.ttl = 30s;
        return (deliver);
    }

    # For everything else, attempt to cache
    set beresp.uncacheable = false;
    set beresp.ttl = 2m;
    return (deliver);
}

Issue 4: Backend Connection Failures

Varnish may fail to reach your backend, resulting in 503 errors for users. This can be caused by backend downtime, network issues, or timeout misconfiguration.

Configuring Backend Health Checks

Define a backend with a health probe so Varnish can detect when it is down and serve stale content instead of erroring.

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 2s;
    .first_byte_timeout = 20s;
    .between_bytes_timeout = 5s;

    .probe = {
        .url = "/health";
        .timeout = 2s;
        .interval = 10s;
        .window = 5;
        .threshold = 3;
    }
}

Setting Up Backend Fallbacks

Configure multiple backends with directors so Varnish can failover automatically.

import directors;

backend server1 {
    .host = "10.0.0.1";
    .port = "8080";
    .probe = {
        .url = "/health";
        .interval = 5s;
        .window = 5;
        .threshold = 2;
    }
}

backend server2 {
    .host = "10.0.0.2";
    .port = "8080";
    .probe = {
        .url = "/health";
        .interval = 5s;
        .window = 5;
        .threshold = 2;
    }
}

sub vcl_init {
    new cluster = directors.round_robin();
    cluster.add_backend(server1);
    cluster.add_backend(server2);
}

sub vcl_recv {
    set req.backend_hint = cluster.backend();
}

Serving Stale Content on Backend Failure

Use grace mode to serve stale content when the backend is unhealthy, ensuring users always get a response.

sub vcl_hit {
    if (obj.ttl >= 0s) {
        return (deliver);
    }

    if (std.healthy(req.backend_hint)) {
        if (obj.ttl + 30s > 0s) {
            return (deliver);
        }
    } else {
        if (obj.ttl + obj.grace > 0s) {
            return (deliver);
        }
    }

    return (miss);
}

Issue 5: Cookie and Session Handling Problems

Varnish does not cache responses to requests with cookies by default. This is correct for authenticated content but problematic when analytics or tracking cookies prevent caching of public pages.

Stripping Unnecessary Cookies

Remove cookies that should not affect caching while preserving session cookies for authenticated users.

sub vcl_recv {
    # Only handle GET and HEAD requests for caching
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Strip analytics and tracking cookies
    set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "_gid=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "_fbp=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "PHPSESSID=[^;]+(; )?", "");

    # Remove trailing semicolons and spaces
    set req.http.Cookie = regsub(req.http.Cookie, ";\s*$", "");

    # If no cookies remain, remove the header entirely
    if (req.http.Cookie == "") {
        unset req.http.Cookie;
    }

    # Pass requests with session cookies to backend
    if (req.http.Cookie ~ "session_id|login_token") {
        return (pass);
    }

    return (hash);
}

Issue 6: Vary Header Conflicts

The Vary response header tells Varnish to store separate cache entries based on request headers. Overly broad Vary headers can cause cache fragmentation, where nearly every request gets its own cache entry.

Diagnosing Vary Issues

Check if your backend sends a Vary: * or Vary: User-Agent header, which effectively defeats caching for all practical purposes.

curl -I http://localhost:8080/my-page | grep -i vary

Normalizing Request Headers

If you must vary on User-Agent, normalize it first to reduce cache variants to a manageable number.

sub vcl_recv {
    # Normalize User-Agent for mobile vs desktop
    if (req.http.User-Agent ~ "(?i)(mobile|android|iphone|ipad)") {
        set req.http.User-Agent = "mobile";
    } else {
        set req.http.User-Agent = "desktop";
    }

    # Normalize Accept-Encoding
    if (req.http.Accept-Encoding) {
        if (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } else if (req.http.Accept-Encoding ~ "br") {
            set req.http.Accept-Encoding = "br";
        } else {
            unset req.http.Accept-Encoding;
        }
    }
}

Removing Problematic Vary Headers

If the backend sends a Vary header you do not want, remove it in vcl_backend_response.

sub backend_response {
    # Remove Vary on User-Agent if we normalized it ourselves
    if (beresp.http.Vary) {
        set beresp.http.Vary = regsub(beresp.http.Vary, "(?i)User-Agent", "");
        set beresp.http.Vary = regsub(beresp.http.Vary, "^,\s*", "");
        if (beresp.http.Vary == "") {
            unset beresp.http.Vary;
        }
    }
}

Issue 7: Memory and Storage Problems

Varnish uses a fixed-size storage backend. When the cache is full, Varnish evicts objects based on LRU (Least Recently Used). If your storage is too small, you may see poor cache hit rates.

Checking Storage Usage

Monitor your storage with varnishstat:

varnishstat -1 | grep -E 'SMA|n_lru_nuked|n_expired'

Key metrics to watch:

• SMA.s0.g_bytes — current bytes in storage
• SMA.s0.g_space — remaining space
• n_lru_nuked — number of objects evicted due to space pressure
• n_expired — number of objects that expired naturally

If n_lru_nuked is high and growing, your cache is too small. Increase the storage size in your Varnish startup configuration.

Configuring Storage Backends

For most workloads, malloc (RAM-based storage) provides the best performance. Configure it in your systemd service file or startup script.

# /etc/systemd/system/varnish.service
[Service]
ExecStart=/usr/sbin/varnishd \
    -a :6081 \
    -f /etc/varnish/default.vcl \
    -s malloc,2g \
    -p feature=+http2 \
    -T 127.0.0.1:6082

For large caches that exceed available RAM, use file-based storage:

ExecStart=/usr/sbin/varnishd \
    -a :6081 \
    -f /etc/varnish/default.vcl \
    -s file,/var/lib/varnish/varnish_storage.bin,10g

Issue 8: VCL Syntax and Runtime Errors

VCL (Varnish Configuration Language) errors can prevent Varnish from starting or cause unexpected behavior at runtime.

Validating VCL Before Deployment

Always test your VCL configuration before reloading it in production.

varnishd -C -f /etc/varnish/default.vcl

This compiles the VCL and prints the generated C code. If there are syntax errors, they will be reported here without affecting the running instance.

Graceful VCL Reloads

Use varnishadm to reload VCL without dropping connections.

# Load new VCL configuration
varnishadm vcl.load new_config /etc/varnish/default.vcl

# Make it active
varnishadm vcl.use new_config

# Optionally discard the old configuration
varnishadm vcl.discard old_config

You can also use the varnishreload utility if available on your system:

varnishreload

Best Practices for Varnish Troubleshooting

Use Diagnostic Headers

Add response headers to help you understand what Varnish did with each request. This is invaluable for debugging in production.

sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    set resp.http.X-Backend = req.backend_hint;
    set resp.http.X-Varnish-Grace = req.http.X-Grace;

    return (deliver);
}

Monitor Key Metrics Continuously

Set up monitoring for these critical Varnish metrics:

• Cache hit ratio — aim for 90% or higher for static content
• Backend request rate — should be significantly lower than client request rate
• Storage utilization — keep below 80% to avoid excessive evictions
• Session queue length — high values indicate backend overload
• Fetch failures — indicates backend connectivity issues

Version Control Your VCL

Treat your VCL files as code. Store them in version control, review changes before deployment, and maintain separate configurations for development, staging, and production environments. This makes it easy to roll back when a change causes issues.

Test with Realistic Traffic

Before deploying VCL changes to production, test them with realistic traffic patterns. Use tools like wrk or vegeta to generate load and verify cache behavior.

# Generate load with wrk
wrk -t4 -c100 -d30s -H "Cookie: session_id=abc123" http://localhost:6081/my-page

# Generate load with vegeta
echo "GET http://localhost:6081/my-page" | vegeta attack -duration=30s -rate=100 | vegeta report

Conclusion

Troubleshooting Varnish Cache requires a systematic approach: observe the behavior with diagnostic tools, identify the root cause in your VCL or backend configuration, and apply targeted fixes. The most common issues—no caching, stale content, cookie interference, and backend failures—all have well-established solutions that you can implement using the techniques covered in this tutorial. By combining proper VCL configuration, health checks, cache purging, and continuous monitoring, you can keep your Varnish instance running efficiently and delivering the performance benefits that make it worth the complexity. Remember to always test VCL changes before deploying them, use diagnostic headers to gain visibility into cache behavior, and monitor your hit ratio and storage metrics to catch problems before your users do.