Introduction to Envoy Proxy
Envoy is a high-performance, open-source edge and service proxy designed for cloud-native applications. Originally developed at Lyft, it is now a graduated project under the Cloud Native Computing Foundation (CNCF). Envoy operates as a Layer 4 (TCP) and Layer 7 (HTTP) proxy, sitting between services to handle network traffic with advanced routing, load balancing, observability, and resilience features.
Unlike traditional proxies that rely on static configuration files and manual reloads, Envoy supports dynamic configuration through xDS APIs, allowing it to adapt in real time as infrastructure changes. This makes it an ideal companion for service meshes like Istio, API gateways, and microservice architectures.
Core Architecture and Components
🚀 Deploy your AI agent in 10 minutes
Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.
Try it free →Understanding Envoy's internal architecture is essential for effective configuration. The proxy is built around a single-process, event-driven model that avoids the overhead of thread synchronization. Key components include:
- Listeners – Define how Envoy accepts incoming connections on specific ports and protocols
- Filters – Processing chains (network and HTTP level) that inspect, transform, and route traffic
- Clusters – Upstream destinations that Envoy can route traffic to, with load balancing and health checking
- Endpoints – Individual host:port pairs within a cluster
- Routes – Rules mapping incoming requests to appropriate clusters based on path, headers, or other criteria
Why Envoy Matters
Envoy has become a cornerstone of modern infrastructure for several compelling reasons:
- Universal Protocol Support – Handles HTTP/1.1, HTTP/2, gRPC, WebSockets, TCP, and UDP traffic transparently
- Rich Observability – Provides detailed metrics, distributed tracing, and structured logging out of the box
- Resilience Features – Includes circuit breaking, automatic retries, timeouts, rate limiting, and outlier detection
- Extensibility – Custom filters can be written in C++, Lua, WebAssembly (Wasm), or configured through external authorization services
- Service Mesh Foundation – Serves as the data plane for Istio, effectively becoming the universal proxy sidecar for Kubernetes workloads
Installation Options
Option 1: Using a Pre-built Binary on Linux
# Download the latest Envoy binary for Linux
curl -L https://github.com/envoyproxy/envoy/releases/download/v1.28.1/envoy-1.28.1-linux-x86_64 -o envoy
# Make it executable
chmod +x envoy
# Verify installation
./envoy --version
Option 2: Docker Container
# Pull the official Envoy Docker image
docker pull envoyproxy/envoy:v1.28.1
# Run Envoy with a mounted configuration
docker run --rm -v $(pwd)/envoy.yaml:/etc/envoy/envoy.yaml \
-p 8080:8080 -p 9901:9901 \
envoyproxy/envoy:v1.28.1 -c /etc/envoy/envoy.yaml
Option 3: Building from Source
# Clone the repository
git clone https://github.com/envoyproxy/envoy.git
cd envoy
# Build using Bazel
bazel build //source/exe:envoy-static
# The binary will be available at bazel-bin/source/exe/envoy-static
First Configuration: A Simple HTTP Proxy
Let's start with a minimal Envoy configuration that proxies incoming HTTP requests to a backend service. This example demonstrates the essential bootstrap configuration structure.
# envoy-minimal.yaml
static_resources:
listeners:
- name: main_listener
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: backend_service
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: backend_cluster
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: backend_cluster
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: backend_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: backend.example.com
port_value: 80
admin:
address:
socket_address:
address: 127.0.0.1
port_value: 9901
Run this configuration with:
./envoy -c envoy-minimal.yaml
This configuration establishes a listener on port 8080, routes all incoming HTTP traffic (any host, prefix "/") to a cluster named backend_cluster, and provides an admin interface on port 9901 for diagnostics and metrics.
Advanced Routing Configuration
Envoy's HTTP routing capabilities are extremely flexible. Below is a more sophisticated configuration showing path-based routing, header matching, and weighted traffic splitting.
# envoy-advanced-routing.yaml
static_resources:
listeners:
- name: main_listener
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
generate_request_id: true
route_config:
name: advanced_routes
virtual_hosts:
- name: main_vhost
domains:
- "api.mycompany.com"
- "api.mycompany.internal"
routes:
# Exact path match
- match:
path: "/health"
route:
cluster: health_check_service
timeout: 2s
# Prefix match with prefix rewrite
- match:
prefix: "/v2/catalog"
route:
cluster: catalog_service_v2
prefix_rewrite: "/catalog"
timeout: 15s
# Regex match
- match:
safe_regex:
regex: "^/users/[0-9]+/profile$"
route:
cluster: user_profile_service
# Header-based routing
- match:
prefix: "/"
headers:
- name: "x-canary"
exact_match: "true"
route:
cluster: canary_cluster
# Weighted traffic split
- match:
prefix: "/checkout"
route:
weighted_clusters:
clusters:
- name: checkout_v1
weight: 90
- name: checkout_v2
weight: 10
total_weight: 100
prefix_rewrite: "/checkout"
# Catch-all route
- match:
prefix: "/"
route:
cluster: main_backend
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: health_check_service
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: health_check_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: health.internal
port_value: 8081
- name: catalog_service_v2
connect_timeout: 3s
type: STRICT_DNS
lb_policy: LEAST_REQUEST
load_assignment:
cluster_name: catalog_service_v2
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: catalog-v2.internal
port_value: 8082
- name: user_profile_service
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: user_profile_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: user-profile.internal
port_value: 8083
- name: canary_cluster
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: canary_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: canary.internal
port_value: 8084
- name: checkout_v1
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: checkout_v1
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: checkout-v1.internal
port_value: 8085
- name: checkout_v2
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: checkout_v2
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: checkout-v2.internal
port_value: 8086
- name: main_backend
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: main_backend
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: main-backend.internal
port_value: 8087
admin:
address:
socket_address:
address: 127.0.0.1
port_value: 9901
Health Checking and Outlier Detection
Envoy supports both active and passive health checking to maintain a pool of healthy upstream endpoints. Combined with outlier detection, this forms a robust resilience strategy.
# envoy-health-checks.yaml — Cluster configuration snippet
clusters:
- name: resilient_backend
connect_timeout: 5s
type: STRICT_DNS
lb_policy: LEAST_REQUEST
# Active health checking
health_checks:
- timeout: 3s
interval: 10s
unhealthy_threshold: 3
healthy_threshold: 1
http_health_check:
host: "resilient-backend.internal"
path: "/health"
method: GET
expected_statuses:
- 200
- 201
# Outlier detection (passive health checking)
outlier_detection:
consecutive_5xx: 5
consecutive_gateway_failure: 3
interval: 30s
base_ejection_time: 60s
max_ejection_time: 300s
max_ejection_percent: 50
success_rate_minimum:
minimum: 80
request_volume: 100
load_assignment:
cluster_name: resilient_backend
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: resilient-backend.internal
port_value: 8080
Active health checks periodically probe endpoints and mark them healthy or unhealthy based on configurable thresholds. Outlier detection monitors live traffic responses and ejects endpoints that exhibit elevated error rates, protecting the system from cascading failures.
Circuit Breaking Configuration
Circuit breakers prevent Envoy from overwhelming upstream services. They limit connections, requests, and pending retries, failing fast when thresholds are exceeded.
# envoy-circuit-breakers.yaml — Cluster with circuit breakers
clusters:
- name: protected_backend
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
circuit_breakers:
thresholds:
- priority: DEFAULT
max_connections: 1000
max_pending_requests: 200
max_requests: 500
max_retries: 50
- priority: HIGH
max_connections: 2000
max_pending_requests: 400
max_requests: 1000
max_retries: 100
load_assignment:
cluster_name: protected_backend
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: protected-backend.internal
port_value: 8080
The DEFAULT priority applies to most traffic, while HIGH priority thresholds can be used for critical requests that require more capacity headroom.
TLS and mTLS Configuration
Envoy provides robust TLS support for both downstream (client-facing) and upstream (backend) connections. Mutual TLS (mTLS) adds authentication by requiring client certificates.
Downstream TLS (Client → Envoy)
# envoy-downstream-tls.yaml — Listener with TLS termination
listeners:
- name: tls_listener
address:
socket_address:
address: 0.0.0.0
port_value: 443
filter_chains:
- filter_chain_match:
server_names: ["secure-api.example.com"]
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/server-cert.pem"
private_key:
filename: "/etc/envoy/certs/server-key.pem"
tls_params:
tls_minimum_protocol_version: TLSv1_2
tls_maximum_protocol_version: TLSv1_3
cipher_suites:
- "ECDHE-ECDSA-AES256-GCM-SHA384"
- "ECDHE-RSA-AES256-GCM-SHA384"
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_tls_http
route_config:
name: tls_routes
virtual_hosts:
- name: secure_backend
domains:
- "secure-api.example.com"
routes:
- match:
prefix: "/"
route:
cluster: secure_backend_cluster
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
Upstream mTLS (Envoy → Backend)
# envoy-upstream-mtls.yaml — Cluster with mTLS
clusters:
- name: secure_backend_cluster
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/client-cert.pem"
private_key:
filename: "/etc/envoy/certs/client-key.pem"
validation_context:
trusted_ca:
filename: "/etc/envoy/certs/ca-cert.pem"
verify_subject_alt_name:
- "secure-backend.internal"
tls_params:
tls_minimum_protocol_version: TLSv1_2
load_assignment:
cluster_name: secure_backend_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: secure-backend.internal
port_value: 8443
Rate Limiting Configuration
Envoy can enforce rate limits using either a local token bucket filter or by integrating with an external rate limiting service. Here is a local rate limiting example at the virtual host level.
# envoy-rate-limiting.yaml — Virtual host with local rate limiting
virtual_hosts:
- name: rate_limited_vhost
domains:
- "api.example.com"
routes:
- match:
prefix: "/"
route:
cluster: backend_service
typed_per_filter_config:
envoy.filters.http.local_ratelimit:
"@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
stat_prefix: http_local_rate_limiter
token_bucket:
max_tokens: 100
tokens_per_fill: 10
fill_interval: 1s
filter_enabled:
runtime_key: "local_rate_limit_enabled"
default_value:
numerator: 100
denominator: HUNDRED
filter_enforced:
runtime_key: "local_rate_limit_enforced"
default_value:
numerator: 100
denominator: HUNDRED
response_headers_to_add:
- key: "x-rate-limited"
value: "true"
status:
code: TooManyRequests
This configuration limits the virtual host to 100 tokens, refilling at 10 tokens per second. When the bucket is empty, Envoy returns a 429 Too Many Requests status. The runtime keys allow dynamic enabling/disabling without configuration reloads.
Observability: Metrics, Logging, and Tracing
Structured Access Logging
# envoy-access-logging.yaml — HttpConnectionManager with access logs
http_connection_manager:
stat_prefix: ingress_http
access_log:
- name: envoy.access_loggers.file
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: "/var/log/envoy/access.log"
format: |
[%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%" up:%UPSTREAM_TLS_VERSION%
route_config:
...
Distributed Tracing with Zipkin
# envoy-tracing.yaml — Tracing configuration snippet
http_connection_manager:
tracing:
provider:
name: envoy.tracers.zipkin
typed_config:
"@type": type.googleapis.com/envoy.config.trace.v3.ZipkinConfig
collector_cluster: zipkin_collector
collector_endpoint: "/api/v2/spans"
collector_endpoint_version: HTTP_JSON
trace_id_128bit: true
shared_span_context: true
# Add the Zipkin collector as a cluster
clusters:
- name: zipkin_collector
connect_timeout: 3s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: zipkin_collector
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: zipkin.internal
port_value: 9411
Prometheus Metrics Scraping
Envoy exposes a rich set of metrics on the admin interface. To scrape with Prometheus, configure a scrape target pointing to the admin port.
# prometheus-scrape.yaml — Prometheus job configuration
scrape_configs:
- job_name: 'envoy'
metrics_path: '/stats/prometheus'
static_configs:
- targets: ['envoy-proxy:9901']
labels:
service: 'envoy-gateway'
The admin endpoint /stats/prometheus provides metrics in Prometheus format, including upstream connection counts, request rates, circuit breaker states, and retry statistics.
Dynamic Configuration with xDS
Envoy's most powerful feature is dynamic configuration via xDS APIs. In production, Envoy typically connects to a control plane that pushes configuration updates in real time. Below is a bootstrap configuration that enables dynamic discovery.
# envoy-dynamic-bootstrap.yaml
dynamic_resources:
lds_config:
api_config_source:
api_type: GRPC
transport_api_version: V3
grpc_services:
- envoy_grpc:
cluster_name: xds_cluster
set_node_on_context_only: true
cds_config:
api_config_source:
api_type: GRPC
transport_api_version: V3
grpc_services:
- envoy_grpc:
cluster_name: xds_cluster
static_resources:
clusters:
- name: xds_cluster
connect_timeout: 5s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
http2_protocol_options:
max_concurrent_streams: 100
load_assignment:
cluster_name: xds_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: control-plane.internal
port_value: 18000
node:
id: "envoy-proxy-001"
cluster: "production-gateway"
metadata:
role: "edge-gateway"
admin:
address:
socket_address:
address: 127.0.0.1
port_value: 9901
With this configuration, Envoy connects to a gRPC control plane at control-plane.internal:18000 and receives Listener Discovery Service (LDS) and Cluster Discovery Service (CDS) updates. The control plane can push routing changes, add or remove clusters, and update TLS certificates without ever restarting Envoy.
WebAssembly (Wasm) Filter Extensions
Envoy supports extending its filter chain with WebAssembly modules, enabling custom logic without recompiling the proxy. Here is how to configure a Wasm HTTP filter.
# envoy-wasm-filter.yaml — HttpConnectionManager with Wasm filter
http_filters:
- name: envoy.filters.http.wasm
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
config:
name: "my_custom_filter"
root_id: "my_root_id"
vm_config:
vm_id: "my_vm_id"
runtime: "envoy.wasm.runtime.v8"
code:
local:
filename: "/etc/envoy/wasm/my_filter.wasm"
allow_precompiled: true
configuration:
"@type": "type.googleapis.com/google.protobuf.StringValue"
value: |
{
"custom_setting": "value",
"feature_flag": true
}
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
Wasm filters can inspect headers, modify request bodies, implement custom authentication, or transform responses. The sandboxed V8 runtime ensures security isolation while maintaining high performance.
TCP Proxy and Non-HTTP Traffic
Envoy is not limited to HTTP. It can proxy raw TCP connections with the same resilience and observability features.
# envoy-tcp-proxy.yaml
static_resources:
listeners:
- name: tcp_listener
address:
socket_address:
address: 0.0.0.0
port_value: 5432
filter_chains:
- filters:
- name: envoy.filters.network.tcp_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
stat_prefix: postgres_tcp_proxy
cluster: postgres_cluster
idle_timeout: 3600s
access_log:
- name: envoy.access_loggers.file
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: "/var/log/envoy/tcp_access.log"
format: "[%START_TIME%] %UPSTREAM_HOST% %BYTES_RECEIVED% %BYTES_SENT% %DURATION%"
clusters:
- name: postgres_cluster
connect_timeout: 5s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: postgres_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: postgres-primary.internal
port_value: 5432
- endpoint:
address:
socket_address:
address: postgres-replica.internal
port_value: 5432
admin:
address:
socket_address:
address: 127.0.0.1
port_value: 9901
This configuration proxies PostgreSQL traffic on port 5432, load balancing across primary and replica instances with full connection-level logging.
Best Practices for Production Deployment
1. Use Dynamic Configuration Wherever Possible
Rely on xDS-based control planes rather than static configuration files. This enables zero-downtime configuration changes and reduces the risk of configuration drift across proxy instances.
2. Implement Graceful Drainage
Configure drain timeouts to allow in-flight requests to complete before Envoy shuts down. This prevents dropped requests during rolling deployments.
# Drain configuration in bootstrap
drain_time: 30s
3. Set Resource Limits
Protect Envoy from memory exhaustion by configuring buffer limits and connection caps.
# Per-route buffer limits
route:
cluster: backend
max_stream_duration:
max_stream_duration: 300s
grpc_max_stream_duration: 300s
per_try_timeout: 15s
4. Enable Comprehensive Observability
Always configure access logs, distributed tracing, and metrics export. The data Envoy collects is invaluable for debugging and capacity planning.
5. Test with Chaos Engineering
Validate circuit breaker thresholds, outlier detection parameters, and retry policies under simulated failure conditions. Use Envoy's fault injection filter to test resilience.
# Fault injection filter for testing
- name: envoy.filters.http.fault
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.fault.v3.FaultInject
abort:
http_status: 503
percentage:
numerator: 10
denominator: HUNDRED
delay:
fixed_delay: 5s
percentage:
numerator: 20
denominator: HUNDRED
6. Run with a Dedicated Admin Interface
Bind the admin interface to localhost only and never expose it publicly. Use it for health checks, metric scraping, and diagnostic queries.
7. Version and Pin Dependencies
Track the Envoy version in your deployment manifests. Use the official Docker images with specific version tags rather than latest to ensure reproducible behavior.
8. Leverage Runtime Feature Flags
Use Envoy's runtime configuration layer to toggle features, adjust thresholds, or enable experimental behavior without full redeployments.
# Runtime configuration file
# envoy-runtime.yaml
runtime:
symlink_root: /srv/runtime/current
subdirectory: envoy
override_subdirectory: envoy_override
Common Debugging Commands
The admin interface provides several endpoints for troubleshooting. Here are the most useful ones:
# Check configuration status
curl http://localhost:9901/config_dump
# List all configured clusters and their health
curl http://localhost:9901/clusters
# View listener status
curl http://localhost:9901/listeners
# Inspect runtime values
curl http://localhost:9901/runtime
# Force a hot restart (if enabled)
curl -X POST http://localhost:9901/restart
# Get Prometheus metrics
curl http://localhost:9901/stats/prometheus
# View server information
curl http://localhost:9901/server_info
Conclusion
Envoy Proxy represents a fundamental shift in how network infrastructure is built and operated in cloud-native environments. Its combination of Layer 7 routing intelligence, robust resilience features, dynamic configuration capabilities, and deep observability makes it an indispensable tool for modern distributed systems. By mastering Envoy's configuration model—from basic HTTP proxying to advanced patterns like circuit breaking, mTLS, rate limiting, and xDS-driven dynamic updates—developers and operators can build service communication layers that are reliable, secure, and adaptable to changing requirements. Whether deployed as an edge gateway, a service mesh sidecar, or a standalone proxy, Envoy delivers the performance and flexibility needed for production-grade microservice architectures.