Fix 'Cannot allocate memory' Error in Linux: Complete Troubleshooting Guide

Understanding the 'Cannot allocate memory' Error

The Cannot allocate memory error (often accompanied by ENOMEM or errno 12) is a critical signal that a process or the kernel itself has failed to obtain the requested virtual memory. It can manifest during a simple fork(), a malloc() call inside your application, or even during seemingly trivial shell operations like listing a directory. This guide will walk you through the internals, diagnostics, and concrete fixes so you can resolve it permanently.

What Is the 'Cannot allocate memory' Error?

At the kernel level, every memory allocation request must be satisfied from one of several pools: userspace virtual memory (backed by RAM or swap), kernel internal slabs, vmalloc space, or dedicated caches. When none of these can fulfill the request, the system call returns -ENOMEM and the C library sets errno to ENOMEM. The typical user-facing message is Cannot allocate memory.

It is crucial to understand that this error is not simply a matter of running out of physical RAM. Linux aggressively overcommits memory by default, allowing processes to request more virtual memory than the system physically has, with the expectation that many pages will never be touched. The error can be triggered by:

• Exhausted swap space combined with high physical memory pressure.
• Per-process limits (ulimit, cgroups) blocking allocation even when system memory is free.
• Kernel memory exhaustion in specific areas (e.g., dentry cache, vmalloc, kernel stack).
• Memory fragmentation where contiguous physical pages are needed but unavailable.
• Strict overcommit accounting (vm.overcommit_memory=2) rejecting a request that exceeds the commit limit.
• Transparent Huge Pages (THP) compaction failures.

Why It Matters

A Cannot allocate memory error is rarely isolated. It can cause:

• Application crashes: A database, web server, or custom daemon suddenly exits, corrupting in-flight transactions.
• Silent OOM (Out-of-Memory) kills: The kernel’s OOM killer terminates processes to free memory, often targeting critical services.
• System instability: The kernel itself may fail to allocate memory for page tables, network buffers, or filesystem operations, leading to hangs or kernel panics.
• Fork bombs: A misconfigured limit allows a user to spawn thousands of processes, exhausting all available memory instantly.
• Containerized workloads: A container hitting its cgroup memory limit sees this error even when the host has ample free RAM, disrupting microservices.

Ignoring the error can lead to a cascading failure across your infrastructure. Proactive understanding is essential.

Diagnosing the Root Cause

Effective troubleshooting requires pinpointing whether the bottleneck is system-wide, per-process, or kernel-internal. Use the following toolkit.

Check System-Wide Memory State

free -h

Look at available (not just free) in the free output. A low available value indicates memory pressure. Also inspect /proc/meminfo directly for deeper counters:

cat /proc/meminfo

Key fields to watch:

• MemAvailable – estimate of how much memory can be allocated without swapping.
• SwapFree – if near zero, the system is at risk when swapping.
• CommitLimit and Committed_AS – shows the overcommit limit and currently committed virtual memory.
• VmallocUsed – kernel virtual memory used for large contiguous allocations.

Identify the Failing Process

Check system logs for OOM killer events or direct error messages:

dmesg | grep -i "out of memory"
journalctl -xe | grep -i "cannot allocate memory"

OOM messages include the killed process name, PID, and a detailed memory dump of total-vm, anon-rss, etc. That is your primary suspect.

If the error is caught by an application, enable its logging or run it under strace to capture the failing syscall:

strace -e trace=memory -f -o strace.log ./your_application
grep ENOMEM strace.log

Inspect Per-Process Limits

Use ulimit (bash) or /proc/PID/limits to see resource caps:

ulimit -a                    # current shell limits
cat /proc/$(pidof your_process)/limits

Pay attention to Max address space (virtual memory, -v) and Max resident set (-m). A low virtual memory limit directly causes ENOMEM on malloc().

Check Cgroup (Container) Memory Limits

For processes inside containers or systemd units, cgroup limits override ulimit. Check the current limit and usage:

# For a process with PID 1234
cat /proc/1234/cgroup
cat /sys/fs/cgroup/memory/$(cat /proc/1234/cgroup | grep memory | cut -d: -f3)/memory.limit_in_bytes
cat /sys/fs/cgroup/memory/$(cat /proc/1234/cgroup | grep memory | cut -d: -f3)/memory.usage_in_bytes

If usage nears the limit, the process receives ENOMEM on attempts to exceed it.

Look for Kernel Memory Exhaustion

Kernel memory issues (dentries, inodes, vmalloc) are visible via:

cat /proc/slabinfo | awk '{print $1, $3*$4}' | sort -k2 -nr | head -20

Large dentry/inode caches can consume gigabytes. Also check /proc/vmallocinfo for fragmentation:

cat /proc/vmallocinfo | wc -l

A massive number of entries suggests severe fragmentation of kernel virtual space.

Common Scenarios and Fixes

Scenario 1: Insufficient RAM or Swap

The simplest cause: physical RAM is genuinely exhausted and swap is either absent or full. The fix involves increasing capacity or reducing load.

Temporary relief: Create a swap file immediately to buy time:

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Make it permanent by adding an entry to /etc/fstab:

/swapfile none swap sw 0 0

Long-term fixes:

• Add physical RAM to the machine.
• Optimize application memory usage (connection pooling, cache size tuning).
• Scale horizontally by distributing workload.

Scenario 2: Per-Process Memory Limits (ulimit)

When a specific application fails but the system has plenty of free RAM, check ulimit -v. For example, a web server might be restricted to 512MB virtual memory.

Fix: Raise the limit in the process’s startup script or systemd unit:

# In /etc/security/limits.conf
myuser          soft    as       unlimited
myuser          hard    as       unlimited

For systemd services, use LimitAS= in the unit file:

[Service]
LimitAS=infinity

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart your_service

Scenario 3: Memory Overcommit and OOM Killer

Linux default overcommit (vm.overcommit_memory=0) allows allocations even when the request seems larger than available RAM+swap. Under heavy demand, this can lead to OOM kills. If you see Cannot allocate memory errors immediately after an OOM kill, the system may be in a tight loop.

Control overcommit behavior:

# View current setting
sysctl vm.overcommit_memory

# 0 - heuristic overcommit (default)
# 1 - always overcommit, never fail unless address space exhausted
# 2 - strict accounting, never overcommit beyond CommitLimit
sudo sysctl -w vm.overcommit_memory=2

When using mode 2, tune vm.overcommit_ratio (percentage of RAM+swap allowed for commitments):

sudo sysctl -w vm.overcommit_ratio=80

To make these permanent, add to /etc/sysctl.conf:

vm.overcommit_memory=2
vm.overcommit_ratio=80

Strict mode prevents sudden OOM kills but may cause Cannot allocate memory earlier. Balance according to workload guarantees.

To protect critical processes from OOM killer, adjust oom_score_adj:

echo -1000 > /proc/$(pidof critical_service)/oom_score_adj

Scenario 4: Kernel Memory Exhaustion (Dentries, Inodes, vmalloc)

A system with plenty of free userspace RAM may still fail allocations inside the kernel. This often appears as Cannot allocate memory in system calls like fork() or file operations.

Clear reclaimable caches:

# Drop dentry and inode caches (safe for running systems)
sync; echo 2 > /proc/sys/vm/drop_caches

Tune kernel parameters for sustained pressure:

# Increase the soft limit for dentry/inode cache reclaim
sysctl -w vm.vfs_cache_pressure=200

If vmalloc space is exhausted (common on 32-bit kernels with large memory), consider switching to a 64-bit kernel, or reduce usage of subsystems that rely on vmalloc (certain drivers).

Scenario 5: Memory Leaks in Applications

A slow memory leak can eventually trigger ENOMEM after days or weeks. Use pmap and valgrind to profile.

# Check memory map of a running process
pmap -x $(pidof leaky_app) | sort -k3 -nr | head -10

# Run application under valgrind for leak detection
valgrind --leak-check=full --log-file=leak.log ./leaky_app

Common culprits:

• Forgotten free() / delete calls.
• Unbounded caches in libraries (e.g., Python object caches, Java heap sizing).
• Circular references preventing garbage collection.

Fix the code, or as a stop-gap, periodically restart the process via cron or systemd timers.

Scenario 6: Container Memory Limits (cgroups)

Docker, Kubernetes, or systemd-nspawn containers often have a memory.limit_in_bytes. Exceeding it triggers Cannot allocate memory inside the container.

Inspect and modify the limit for a Docker container:

# Check current limit
docker inspect your_container | grep -i memory

# Update limit (requires restart)
docker update --memory 2g --memory-swap 2g your_container

For Kubernetes pods, adjust resources.limits.memory in the pod spec and reapply.

Scenario 7: Transparent Huge Pages (THP) Fragmentation

THP can cause allocation failures when the kernel tries to compact memory to form huge pages and fails, especially under high load. Symptoms include ENOMEM during mmap() with MAP_HUGETLB or even normal allocations if compaction is aggressive.

Disable THP entirely:

echo never > /sys/kernel/mm/transparent_hugepage/enabled

Or set to madvise so only applications that explicitly request THP are affected:

echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

Make persistent via rc.local or kernel command line parameter transparent_hugepage=never.

Scenario 8: Fork Bomb or Process Spawn Limits

A fork bomb consumes all memory by spawning countless processes. The error appears as Cannot allocate memory for fork(). Prevent by limiting user processes via ulimit -u or cgroup pids.max.

# Set max user processes in limits.conf
myuser    hard    nproc   2000

# Or via systemd user slice
systemctl set-property user-1000.slice TasksMax=2000

How to Use Monitoring and Prevention

Proactive monitoring catches memory pressure before users experience errors. Deploy node_exporter with Prometheus and set alerts on:

• node_memory_MemAvailable_bytes dropping below a threshold.
• Rate of OOM kills: node_vmstat_oom_kill increments.
• Swap usage percentage: node_memory_SwapFree_bytes / node_memory_SwapTotal_bytes.
• Container memory usage nearing limits.

A simple script can log warnings when MemAvailable is low:

#!/bin/bash
THRESHOLD_MB=512
AVAILABLE_MB=$(awk '/MemAvailable/ {print $2}' /proc/meminfo | awk '{print int($1/1024)}')
if [ "$AVAILABLE_MB" -lt "$THRESHOLD_MB" ]; then
    echo "Low memory warning: $AVAILABLE_MB MB available" | systemd-cat -p warning
fi

Run this via cron or a systemd timer every minute.

Best Practices to Avoid 'Cannot allocate memory' Errors

• Right-size swap: Even on RAM-rich machines, a small swap file (2–4 GB) provides safety against sudden allocation spikes.
• Set realistic limits: Use ulimit -v and cgroup memory limits that match your application’s proven working set plus a 20% buffer.
• Monitor committed memory: Watch Committed_AS vs CommitLimit; when they approach each other, you’re at risk of OOM even with free RAM.
• Enable strict overcommit for predictable workloads: Databases and latency-sensitive services benefit from vm.overcommit_memory=2 to catch errors early without OOM kills.
• Profile memory usage regularly: Run valgrind or heaptrack in staging to catch leaks before production.
• Keep the kernel updated: Memory management improvements and fixes for fragmentation arrive with newer kernels.
• Disable THP if you observe allocation failures: Many databases recommend transparent_hugepage=never to avoid latency and allocation issues.
• Plan capacity: Use historical data from sar -r or Prometheus to forecast memory needs and scale ahead of demand.

Conclusion

The Cannot allocate memory error is a multifaceted symptom that can originate from userspace, kernel internals, or configuration limits. By systematically checking system-wide memory, per-process limits, kernel caches, and container boundaries, you can isolate the exact bottleneck. Applying the appropriate fix—whether it’s adding swap, tuning overcommit, adjusting ulimit, clearing caches, or fixing a leak—restores stability and prevents recurrence. Combine these reactive steps with proactive monitoring and capacity planning, and you’ll turn a cryptic error into a manageable, well-understood signal in your Linux environment.