Ubuntu Performance Tuning for Web Servers
Ubuntu is one of the most popular Linux distributions for hosting web servers, but a default installation is optimized for general-purpose use rather than high-traffic web workloads. Performance tuning involves adjusting kernel parameters, network stack settings, file descriptor limits, and application-level configurations to squeeze maximum throughput and minimum latency out of your hardware. This tutorial walks you through the essential tuning steps every web server administrator should know.
Why Performance Tuning Matters
An untuned Ubuntu server can handle moderate traffic, but under load it will quickly reveal bottlenecks: dropped connections, slow response times, exhausted file descriptors, and high memory pressure. Proper tuning delivers several key benefits:
- Higher concurrency: More simultaneous client connections without dropping requests.
- Lower latency: Faster TCP handshakes and reduced time-to-first-byte.
- Better resource utilization: Efficient use of CPU, memory, and network buffers.
- Improved stability: Fewer outages caused by resource exhaustion during traffic spikes.
- Cost savings: A tuned server can handle more traffic, delaying the need to scale horizontally.
Prerequisites
Before applying any changes, ensure you have:
- An Ubuntu 20.04 or 22.04 server with root or sudo access.
- A backup of your current configuration files.
- A monitoring solution (such as Netdata, Prometheus, or at minimum
htopanddstat) to measure the impact of changes. - A staging environment to test changes before deploying to production.
1. Tuning the Kernel Network Stack
The Linux kernel exposes hundreds of tunable parameters through /proc/sys. The most impactful for web servers relate to TCP networking, connection tracking, and backlog queues.
1.1 Editing sysctl Parameters
All kernel tunables are managed via the sysctl command and persisted in /etc/sysctl.conf or individual files under /etc/sysctl.d/. Create a dedicated file for your web server tuning:
sudo nano /etc/sysctl.d/99-webserver.conf
Add the following configuration, which is a solid baseline for a high-traffic web server:
# Increase TCP max buffer size
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
# Increase TCP read/write buffers
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# Increase connection backlog queue
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
# Enable TCP Fast Open
net.ipv4.tcp_fastopen = 3
# Reuse sockets in TIME_WAIT state
net.ipv4.tcp_tw_reuse = 1
# Decrease FIN_WAIT timeout
net.ipv4.tcp_fin_timeout = 15
# Keepalive settings
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15
# Increase local port range for outbound connections
net.ipv4.ip_local_port_range = 10000 65535
# Increase connection tracking table size
net.netfilter.nf_conntrack_max = 1048576
# Enable BBR congestion control
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
# Disable IPv6 if not used (reduces overhead)
# net.ipv6.conf.all.disable_ipv6 = 1
Apply the changes immediately:
sudo sysctl -p /etc/sysctl.d/99-webserver.conf
Verify that BBR congestion control is active:
sysctl net.ipv4.tcp_congestion_control
The output should show net.ipv4.tcp_congestion_control = bbr. BBR (Bottleneck Bandwidth and Round-trip propagation time) is a modern congestion control algorithm developed by Google that significantly improves throughput on lossy or high-latency networks compared to the default CUBIC algorithm.
1.2 Understanding the Backlog Queue
The somaxconn parameter defines the maximum number of connections that can be queued for acceptance by the listening socket. If your web server receives bursts of connections and somaxconn is too low, the kernel will drop new connections before your application can accept them. Setting it to 65535 ensures the kernel can buffer a large number of pending connections.
Similarly, tcp_max_syn_backlog controls the queue of connections that have received a SYN packet but not yet completed the three-way handshake. During SYN floods or traffic spikes, a larger backlog prevents legitimate connections from being dropped.
2. Increasing File Descriptor Limits
On Linux, every open socket, file, and pipe consumes a file descriptor. The default per-process limit is often 1024, which is far too low for a web server handling thousands of concurrent connections. You need to raise both the soft and hard limits.
2.1 System-Wide File Descriptor Limit
Edit the system-wide limits file:
sudo nano /etc/sysctl.conf
Add or modify this line:
fs.file-max = 2097152
Apply the change:
sudo sysctl -p
2.2 Per-User Limits with PAM
Edit the limits configuration file:
sudo nano /etc/security/limits.conf
Add the following lines at the end (replace www-data with the user your web server runs as, or use * for all users):
* soft nofile 1048576
* hard nofile 1048576
* soft nproc 65535
* hard nproc 65535
root soft nofile 1048576
root hard nofile 1048576
If your system uses systemd (which Ubuntu does by default), you also need to configure the systemd limits. Edit the systemd user configuration:
sudo nano /etc/systemd/system.conf
Add or uncomment:
DefaultLimitNOFILE=1048576
DefaultLimitNPROC=65535
Do the same in /etc/systemd/user.conf, then reload systemd:
sudo systemctl daemon-reexec
Restart your web server and verify the new limits:
cat /proc/$(pgrep -o nginx)/limits | grep "Max open files"
3. Tuning Nginx for High Traffic
Nginx is the most common web server on Ubuntu. Its default configuration is conservative. Here is how to tune it for production workloads.
3.1 Worker Processes and Connections
Edit the main Nginx configuration:
sudo nano /etc/nginx/nginx.conf
Adjust the worker settings:
worker_processes auto;
worker_rlimit_nofile 1048576;
events {
worker_connections 65535;
multi_accept on;
use epoll;
}
The worker_processes auto directive tells Nginx to spawn one worker per CPU core. The worker_connections setting defines how many simultaneous connections each worker can handle. With multi_accept on, each worker will accept all new connections at once rather than one at a time.
3.2 HTTP Block Tuning
Within the http block, add or modify these directives:
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 30;
keepalive_requests 1000;
reset_timedout_connection on;
client_body_timeout 10;
client_header_timeout 10;
send_timeout 10;
# Buffer sizes
client_body_buffer_size 16k;
client_header_buffer_size 1k;
large_client_header_buffers 4 8k;
client_max_body_size 8m;
# Gzip compression
gzip on;
gzip_min_length 256;
gzip_comp_level 5;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
# Open file cache
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
}
Test the configuration and reload:
sudo nginx -t && sudo systemctl reload nginx
4. Tuning PHP-FPM for Dynamic Content
If you serve PHP applications, PHP-FPM is often the primary consumer of CPU and memory. The default process manager settings are rarely optimal.
4.1 Configuring the Process Manager
Edit your PHP-FPM pool configuration (path varies by PHP version):
sudo nano /etc/php/8.1/fpm/pool.d/www.conf
Use the dynamic process manager with carefully calculated values:
[www]
pm = dynamic
pm.max_children = 120
pm.start_servers = 12
pm.min_spare_servers = 6
pm.max_spare_servers = 18
pm.max_requests = 1000
To calculate pm.max_children, you need to know how much memory each PHP worker consumes. Run this command to check average memory usage per process:
ps --no-headers -o "rss,cmd" -C php-fpm8.1 | awk '{ sum+=$1 } END { printf ("Average memory per process: %.2f MB\n", sum/NR/1024) }'
Then use this formula: pm.max_children = (Total RAM reserved for PHP) / (Memory per process in MB). For example, if you have 4GB reserved for PHP and each process uses 30MB, set pm.max_children to approximately 130.
The pm.max_requests setting prevents memory leaks by recycling workers after they handle 1000 requests. Restart PHP-FPM to apply:
sudo systemctl restart php8.1-fpm
5. Optimizing Disk I/O and Filesystem
Disk I/O can become a bottleneck for web servers that log heavily or serve many static files. Several kernel and filesystem tweaks can help.
5.1 I/O Scheduler
For SSD and NVMe drives, the none (or noop) scheduler is optimal because these devices have their own internal scheduling. For traditional HDDs, deadline is preferred. Check your current scheduler:
cat /sys/block/sda/queue/scheduler
Change it temporarily:
echo none | sudo tee /sys/block/sda/queue/scheduler
To make it persistent, create a udev rule:
sudo nano /etc/udev/rules.d/60-io-scheduler.rules
ACTION=="add|change", KERNEL=="sd[a-z]|nvme[0-9]n[0-9]", ATTR{queue/scheduler}="none"
5.2 Reducing Swappiness
The vm.swappiness parameter controls how aggressively the kernel swaps anonymous memory pages to disk. For web servers, you want the kernel to prefer keeping application data in RAM:
sudo sysctl vm.swappiness=10
Make it persistent by adding to your sysctl file:
echo "vm.swappiness=10" | sudo tee -a /etc/sysctl.d/99-webserver.conf
5.3 Disabling Access Time Updates
Mounting filesystems with the noatime option prevents the kernel from writing to disk every time a file is read, reducing I/O overhead:
sudo nano /etc/fstab
Add noatime to the mount options for your root or data partition:
UUID=your-uuid / ext4 defaults,noatime,errors=remount-ro 0 1
Reboot or remount to apply:
sudo mount -o remount /
6. Tuning MySQL/MariaDB for Web Backends
Database performance is often the single biggest factor in web application response time. Here are the most impactful settings for a web server database.
6.1 InnoDB Buffer Pool
The InnoDB buffer pool caches table data and indexes in memory. For a dedicated database server, allocate 60-70% of total RAM. For a combined web and database server, allocate 20-40%:
sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
[mysqld]
innodb_buffer_pool_size = 2G
innodb_buffer_pool_instances = 4
innodb_log_file_size = 512M
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
innodb_io_capacity = 2000
innodb_io_capacity_max = 4000
max_connections = 300
thread_cache_size = 50
table_open_cache = 4000
table_definition_cache = 2000
query_cache_type = 0
query_cache_size = 0
tmp_table_size = 64M
max_heap_table_size = 64M
The innodb_flush_log_at_trx_commit = 2 setting improves write performance by flushing logs once per second instead of on every commit. This introduces a small risk of data loss during a crash, so use 1 if data integrity is critical. The O_DIRECT flush method bypasses the operating system buffer cache, preventing double buffering.
Restart MySQL to apply:
sudo systemctl restart mysql
7. Enabling TCP BBR and Queue Disciplines
We briefly mentioned BBR in the sysctl section, but it deserves deeper explanation. BBR models the network bottleneck to maximize throughput and minimize buffer bloat. Combined with the fq (fair queuing) discipline, it provides excellent performance for web traffic.
Verify BBR is loaded as a kernel module:
lsmod | grep bbr
If it is not listed, load it:
sudo modprobe tcp_bbr
To ensure it loads at boot, add it to the modules file:
echo "tcp_bbr" | sudo tee -a /etc/modules-load.d/modules.conf
8. Security Limits and Connection Tracking
If you use a firewall with connection tracking (such as UFW with iptables or nftables), the conntrack table can become a bottleneck. Monitor the current usage:
sudo cat /proc/sys/net/netfilter/nf_conntrack_count
sudo cat /proc/sys/net/netfilter/nf_conntrack_max
If the count approaches the max, increase the table size and reduce timeout values:
sudo nano /etc/sysctl.d/99-webserver.conf
net.netfilter.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_tcp_timeout_established = 3600
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 30
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 30
Apply the changes:
sudo sysctl -p /etc/sysctl.d/99-webserver.conf
9. Monitoring and Benchmarking
Tuning without measurement is guesswork. You need tools to establish baselines and verify improvements.
9.1 Real-Time Monitoring
Install a few essential monitoring tools:
sudo apt update
sudo apt install htop iotop dstat sysstat
Enable sar to collect historical performance data:
sudo systemctl enable --now sysstat
View network statistics with sar:
sar -n DEV 1
9.2 Load Testing
Use wrk or hey to benchmark your server before and after tuning:
sudo apt install wrk
Run a basic load test:
wrk -t4 -c1000 -d30s http://localhost/
This runs a 30-second test with 4 threads and 1000 concurrent connections. Compare requests per second and latency percentiles before and after applying your tuning changes.
10. Best Practices
- Change one thing at a time: Apply a single tuning change, measure the impact, then move to the next. This makes it easy to identify which change caused any regression.
- Document every change: Keep a changelog of what you modified, the original value, the new value, and the rationale. This is invaluable for troubleshooting.
- Test in staging first: Never apply kernel or web server tuning directly to production without testing in an environment that mirrors your production load.
- Monitor continuously: Tuning is not a one-time activity. Traffic patterns change, applications evolve, and hardware ages. Set up alerting for key metrics like CPU load, memory usage, disk I/O, and connection counts.
- Profile before optimizing: Use tools like
perf,strace, andtcpdumpto identify the actual bottleneck before applying changes. Optimizing the wrong component wastes effort. - Keep your system updated: Kernel updates often include performance improvements. Regularly apply security and feature updates, but test them in staging first.
- Use a reverse proxy cache: Nginx's
proxy_cacheor a dedicated cache like Varnish can dramatically reduce backend load by serving cached responses directly. - Enable HTTP/2 or HTTP/3: These protocols offer multiplexing and header compression, reducing connection overhead and improving page load times for modern clients.
- Right-size your hardware: No amount of tuning can compensate for insufficient RAM or CPU. Monitor resource utilization and upgrade hardware when you consistently operate above 70% capacity.
Conclusion
Ubuntu performance tuning for web servers is a systematic process that spans the kernel network stack, file descriptor limits, web server configuration, application runtimes, database settings, and disk I/O. By applying the changes in this tutorial incrementally and measuring their impact with proper monitoring and load testing tools, you can significantly increase the number of concurrent connections your server handles, reduce response latency, and improve overall stability under load. Remember that tuning is an ongoing practice rather than a one-time task. As your traffic grows and your application evolves, revisit these settings regularly, profile new bottlenecks, and adjust your configuration to match the current demands of your workload.