Fix 'Cannot allocate memory' Error in Linux

Understanding the 'Cannot allocate memory' Error in Linux

The "Cannot allocate memory" error (often surfacing as ENOMEM or errno 12) is one of the most frustrating yet common issues developers and system administrators encounter on Linux systems. It can appear during process forking, memory mapping, thread creation, or even when executing simple shell commands. This tutorial will walk you through the complete diagnostic and resolution workflow.

What the Error Actually Means

At its core, this error indicates that the operating system cannot fulfill a memory allocation request. This doesn't always mean the system is completely out of physical RAM. The kernel may refuse an allocation request due to several distinct resource constraints:

Exhausted physical RAM + swap space: No pages available anywhere
Process-specific limits hit: ulimit restrictions on virtual memory, data segment, or max processes
Kernel memory exhaustion: Particularly in the fork() path where the kernel needs to allocate a process control block (task_struct) and kernel stack
Memory fragmentation: Contiguous physical pages unavailable for a kernel allocation
cgroup memory limits: Container or systemd service unit memory caps exceeded
Overcommit policy denial: The kernel's overcommit handler (vm.overcommit_memory) rejected the allocation based on heuristics

Why This Error Matters

Ignoring this error leads to cascading failures: databases crash during query execution, web servers drop requests, container orchestration systems evict pods, and critical system daemons like sshd or cron may silently die. In production environments, a single ENOMEM event can trigger an outage that takes down an entire service tier. Understanding the root cause—not just the symptom—is essential for building resilient systems.

Diagnosing the Root Cause

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Step 1: Check System-Wide Memory Statistics

Start with a broad view. Use free and /proc/meminfo to understand the current state:

# Get a human-readable summary
free -h

# Detailed breakdown from the kernel
cat /proc/meminfo

# Watch memory pressure in real time (every 2 seconds)
watch -n 2 'free -h; echo "---"; cat /proc/meminfo | grep -E "MemAvailable|SwapFree|CommitLimit|Committed_AS"'

Key fields to watch in /proc/meminfo:

MemAvailable — The amount of memory available for new allocations without swapping
SwapFree — If this is near zero and swap usage is high, the system is under severe pressure
Committed_AS vs CommitLimit — If committed bytes approach the limit, the kernel may deny new allocations depending on overcommit policy
VmallocUsed — Exhaustion here can cause kernel module loading or large kernel allocations to fail

Step 2: Identify the Culprit Process

Find which process is consuming excessive memory or triggering the allocation failures:

# Sort processes by resident memory (RSS)
ps aux --sort=-rss | head -20

# Alternative: top in batch mode sorted by memory
top -b -n 1 -o %MEM | head -20

# Check per-process memory maps for a suspicious PID
cat /proc/PID/status | grep -E "VmRSS|VmSize|VmPeak|VmData|VmStk"
cat /proc/PID/smaps_rollup

# Count processes per user (fork bombs often cause ENOMEM)
ps aux | awk '{print $1}' | sort | uniq -c | sort -rn | head -10

Step 3: Check Process Limits (ulimit)

A process hitting its own resource ceiling will see ENOMEM even if the system has free memory:

# Check limits for a running process (replace PID)
cat /proc/PID/limits

# Common restrictive limits:
# Max processes (-u) — fork() fails with EAGAIN/ENOMEM
# Max virtual memory (-v) — mmap/malloc fail
# Max data segment (-d) — brk/sbrk fail

# Check current shell limits
ulimit -a

# For a specific running service (systemd)
systemctl show myservice.service | grep -i limit

Step 4: Inspect cgroup / Container Limits

If running inside Docker, Kubernetes, or systemd with MemoryMax, cgroup limits are the most common cause:

# For a Docker container
docker inspect CONTAINER_ID | grep -i memory

# Check cgroup memory stats directly
cat /sys/fs/cgroup/memory/docker/CONTAINER_ID/memory.limit_in_bytes
cat /sys/fs/cgroup/memory/docker/CONTAINER_ID/memory.usage_in_bytes
cat /sys/fs/cgroup/memory/docker/CONTAINER_ID/memory.events

# For systemd services with memory limits
systemctl show myservice.service | grep -E "MemoryCurrent|MemoryMax|MemoryLimit"

# Check Kubernetes pod cgroup (find the cgroup path first)
cat /sys/fs/cgroup/memory/kubepods.slice/kubepods-besteffort.slice/.../memory.limit_in_bytes

Step 5: Analyze the Kernel Overcommit Policy

The kernel's memory overcommit behavior is controlled by vm.overcommit_memory:

# Read current overcommit settings
cat /proc/sys/vm/overcommit_memory
cat /proc/sys/vm/overcommit_ratio
cat /proc/sys/vm/overcommit_kbytes

# Interpretation:
# overcommit_memory = 0: heuristic overcommit (default) — kernel estimates availability
# overcommit_memory = 1: always overcommit — malloc almost never fails
# overcommit_memory = 2: strict accounting — allocations limited to swap + ratio% of RAM

# Check the current commit limit and usage
grep -E "CommitLimit|Committed_AS" /proc/meminfo

Fixing the Error

Fix 1: Free Up Memory Immediately

The fastest relief when the system is genuinely out of memory:

# Drop page cache, dentries, and inodes (safe, reclaimable)
echo 3 > /proc/sys/vm/drop_caches

# Sync first to avoid data loss, then drop
sync; echo 3 > /proc/sys/vm/drop_caches

# Kill the largest memory consumer (careful in production!)
# Identify the biggest RSS process
ps aux --sort=-rss | head -5
kill -15 PID   # graceful first
kill -9 PID    # force if unresponsive

# If a specific service is bloated, restart it
systemctl restart postgresql.service

Fix 2: Increase Swap Space

Adding swap gives the kernel more headroom, especially important on systems with spinning disks where swap performance is acceptable for infrequently accessed pages:

# Create a 4GB swap file
dd if=/dev/zero of=/swapfile bs=1M count=4096
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile

# Make it permanent
echo '/swapfile none swap sw 0 0' >> /etc/fstab

# Verify
swapon --show
free -h

Fix 3: Adjust Process ulimit Values

Raise limits for processes that legitimately need more memory:

# Temporarily for current shell
ulimit -v unlimited
ulimit -u 65535

# Permanently via limits.conf
# Add to /etc/security/limits.conf:
# myuser    soft    nproc     65535
# myuser    hard    nproc     65535
# myuser    soft    as        unlimited
# myuser    hard    as        unlimited

# For systemd services, override in the unit file or drop-in:
# /etc/systemd/system/myservice.service.d/limits.conf
[Service]
LimitNOFILE=65535
LimitNPROC=65535
LimitAS=infinity

Fix 4: Modify Kernel Overcommit Policy

If strict overcommit (mode 2) is causing premature allocation failures, adjust it:

# Temporarily switch to heuristic mode
echo 0 > /proc/sys/vm/overcommit_memory

# Or allow full overcommit (use cautiously)
echo 1 > /proc/sys/vm/overcommit_memory

# For permanent change, add to /etc/sysctl.conf or /etc/sysctl.d/99-overcommit.conf:
vm.overcommit_memory = 0
vm.overcommit_ratio = 50    # percentage of RAM for commit accounting in mode 2
vm.overcommit_kbytes = 0    # or set explicit kbytes

# Apply without reboot
sysctl -p /etc/sysctl.d/99-overcommit.conf

Fix 5: Tune the OOM Killer Behavior

When memory is genuinely exhausted, the Out-Of-Memory (OOM) killer selects a victim. You can influence its decisions:

# Check current OOM score of processes
cat /proc/PID/oom_score
cat /proc/PID/oom_score_adj

# Protect a critical process from being killed (negative adj)
echo -1000 > /proc/PID/oom_score_adj

# Make a process more likely to be killed (positive adj)
echo 1000 > /proc/PID/oom_score_adj

# Disable OOM killer entirely for a process (DANGER: kernel may panic)
echo -1000 > /proc/PID/oom_score_adj   # effectively immune

# Check system-wide OOM control
cat /proc/sys/vm/panic_on_oom   # 0 = OOM kills process, 1 = kernel panic

Fix 6: Resolve Memory Leaks in Application Code

If a specific process grows unbounded, the fix must happen at the application level. Here are diagnostic techniques for developers:

# Attach with strace to see failing syscalls
strace -p PID -e trace=mmap,brk,madvise -o strace_output.log

# Use valgrind for offline leak detection (restart process under valgrind)
valgrind --leak-check=full --track-origins=yes --log-file=valgrind.log ./myapp

# For Java applications, check heap usage
jstat -gcutil PID 1000 10
jmap -heap PID
jmap -histo:live PID | head -30

# For Node.js, check heap with inspector
node --inspect --expose-gc myapp.js
# Then in Chrome DevTools: Memory > Take Heap Snapshot

# For Python, use tracemalloc embedded in the app:
import tracemalloc
tracemalloc.start()
# ... after suspected leak ...
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics('lineno')[:10]:
    print(stat)

Fix 7: Container-Specific Solutions

For Docker/Kubernetes workloads:

# Docker: Increase container memory limit
docker update --memory 2g --memory-swap 3g CONTAINER_ID

# Or recreate with higher limits
docker run -d --memory 2g --memory-swap 3g --name myapp myimage

# Kubernetes: Adjust resource limits in the pod spec
# Edit the deployment or pod YAML:
resources:
  limits:
    memory: "2Gi"
  requests:
    memory: "1Gi"

# Apply the change
kubectl apply -f deployment.yaml

# Check if OOMKilled events occurred
kubectl describe pod POD_NAME | grep -i oom
kubectl get events --field-selector reason=OOMKilling

Fix 8: Address Fork Bomb Scenarios

A fork bomb (rapid uncontrolled process creation) exhausts kernel memory for process tables, not necessarily user memory:

# Detect a fork bomb: look for an explosion of process count
ps aux | wc -l
# Compare with normal baseline

# Limit processes per user in /etc/security/limits.conf
*               hard    nproc           4096

# systemd also provides process limits per service
# In the unit file:
[Service]
TasksMax=512

# Use cgroups pids controller to limit forks
echo 512 > /sys/fs/cgroup/pids/user.slice/user-1000.slice/pids.max

Best Practices for Prevention

1. Implement Proactive Monitoring

Set up alerts before ENOMEM actually occurs:

# Script for Nagios / Sensu / custom monitoring
#!/bin/bash
AVAILABLE=$(grep MemAvailable /proc/meminfo | awk '{print $2}')
THRESHOLD_MB=512
THRESHOLD_KB=$((THRESHOLD_MB * 1024))

if [ "$AVAILABLE" -lt "$THRESHOLD_KB" ]; then
    echo "CRITICAL: Available memory ${AVAILABLE}KB < ${THRESHOLD_KB}KB"
    exit 2
fi
echo "OK: Available memory ${AVAILABLE}KB"
exit 0

2. Configure systemd Service Memory Limits

Prevent a single runaway service from consuming all system memory:

# Example systemd service unit with memory protection
[Service]
MemoryMax=1G
MemoryHigh=800M
MemorySwapMax=500M
TasksMax=256

# MemoryHigh triggers throttling before MemoryMax hard limit
# MemorySwapMax limits swap usage specifically

3. Use cgroup-aware Application Design

Modern applications should detect their cgroup limits and adapt behavior:

// Example C snippet: read cgroup memory limit
#include 
#include 

long get_cgroup_memory_limit() {
    FILE *f = fopen("/sys/fs/cgroup/memory/memory.limit_in_bytes", "r");
    if (!f) return -1;
    long limit;
    fscanf(f, "%ld", &limit);
    fclose(f);
    return limit;
}

// Use this to size internal caches or decide to refuse requests

4. Enable Early OOM Daemons

Tools like earlyoom or nohang proactively kill processes before the system becomes completely unresponsive:

# Install earlyoom on Debian/Ubuntu
apt install earlyoom

# Configure thresholds in /etc/default/earlyoom
# Kill when available memory < 10% and swap < 10%
earlyoom -m 10 -s 10

# For systemd-based systems, enable the service
systemctl enable --now earlyoom

5. Perform Regular Memory Profiling in CI/CD

Catch memory regressions before they reach production:

# Example CI pipeline step: run app under memory limit, record RSS
#!/bin/bash
# Start app in background with a virtual memory limit
ulimit -v 1048576  # 1GB virtual memory max
./myapp &
PID=$!
sleep 30  # let it stabilize

# Sample RSS every second for 60 seconds
for i in $(seq 1 60); do
    RSS=$(grep VmRSS /proc/$PID/status | awk '{print $2}')
    echo "$(date +%s) $RSS" >> rss_log.txt
    sleep 1
done

kill $PID

# Fail CI if max RSS exceeds threshold (e.g., 500MB = 512000 KB)
MAX_RSS=$(awk '{print $2}' rss_log.txt | sort -n | tail -1)
if [ "$MAX_RSS" -gt 512000 ]; then
    echo "FAIL: Max RSS ${MAX_RSS}KB exceeds limit"
    exit 1
fi
echo "PASS: Max RSS ${MAX_RSS}KB within limit"

6. Understand and Tune Kernel Memory Reclaim

Kernel parameters that affect how aggressively memory is reclaimed:

# View current vm settings
sysctl -a | grep -E "vm.swappiness|vm.vfs_cache_pressure|vm.min_free_kbytes|vm.watermark"

# Common tuning (apply via sysctl)
vm.swappiness = 10          # reduce tendency to swap (default 60)
vm.vfs_cache_pressure = 50  # reclaim dentry/inode caches less aggressively (default 100)
vm.min_free_kbytes = 262144 # ensure 256MB always free for critical allocations
vm.watermark_scale_factor = 10 # reduce watermark scaling to reclaim sooner

# Apply persistently
echo "vm.swappiness=10" >> /etc/sysctl.d/99-memory-tuning.conf
echo "vm.vfs_cache_pressure=50" >> /etc/sysctl.d/99-memory-tuning.conf
sysctl --system

Real-World Troubleshooting Scenarios

Scenario A: "Cannot allocate memory" during SSH login

This often indicates kernel memory exhaustion from too many processes (fork bomb or zombie accumulation):

# Check process count
ps aux | wc -l
cat /proc/loadavg

# Check for zombies
ps aux | grep 'Z'

# If process count is absurdly high (thousands), identify the parent
pstree -p | head -50

# Temporarily raise kernel pid limit
echo 4194304 > /proc/sys/kernel/pid_max

# Kill the offending parent process tree
kill -9 -PARENT_PID   # negative PID = process group

Scenario B: malloc() fails in a container despite free host memory

The container's cgroup memory limit is hit. Verify and adjust:

# Inside container, check cgroup limit
cat /sys/fs/cgroup/memory/memory.limit_in_bytes
cat /sys/fs/cgroup/memory/memory.usage_in_bytes

# If limit is 1GB and usage is 1GB, allocation fails
# Solution: increase container limit (see Fix 7 above)
# Or reduce application memory usage

# Quick workaround: disable cgroup memory accounting (not recommended for production)
# This requires host access and is a temporary debug measure

Scenario C: Intermittent ENOMEM during high traffic spikes

This suggests memory pressure from transient allocations (network buffers, connection tracking):

# Check network-related memory
cat /proc/net/sockstat
# Look at TCP memory pressure
cat /proc/sys/net/ipv4/tcp_mem

# Increase network buffer limits
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216

# Increase connection tracking table size
sysctl -w net.netfilter.nf_conntrack_max=1048576

# For high-concurrency servers, pre-allocate hugepages for buffers
echo 1024 > /proc/sys/vm/nr_hugepages

Conclusion

The "Cannot allocate memory" error in Linux is a multifaceted issue that demands systematic diagnosis rather than blind restarts. By working through the diagnostic layers—system-wide memory, per-process limits, cgroup constraints, kernel overcommit policy, and application-level leaks—you can pinpoint the exact bottleneck. The fixes range from immediate operational interventions like dropping caches or increasing swap, to architectural changes such as configuring service memory limits, tuning kernel reclaim parameters, and implementing proactive monitoring. The best defense is a layered prevention strategy: set resource boundaries via cgroups and systemd, monitor memory pressure continuously, profile application memory in CI pipelines, and maintain sufficient swap as a safety net. With these practices in place, you transform ENOMEM from a mysterious outage trigger into a well-understood, manageable condition.