Understanding Swap in Linux: What Happens When Linux Runs Out of RAM

Linux treats available memory differently than most tools report. The kernel keeps recently used files in RAM as page cache. This memory is reclaimable instantly. free subtracts it from what it calls “available” memory because the kernel can drop it whenever a process needs it. 596 MB of available does not mean you are nearly out of RAM. It means the kernel has already freed what it can.

Two Kinds of Memory

Every process uses anonymous memory or file-backed memory. Anonymous memory is the heap and stack of a process. It has no file on disk. When the kernel needs RAM for something else, it cannot drop anonymous memory to disk; it must write it to swap. File-backed memory includes page cache, shared libraries, and anything mmap’d from a file. The kernel can drop this instantly because it can always re-read from the original file.

Check the watermarks on your system:

grep -A 3 "watermark" /proc/zoneinfo | head -20

swappiness

The swappiness setting controls how aggressively the kernel swaps out anonymous memory. It ranges from 0 to 100. The default on this VPS is 60:

cat /proc/sys/vm/swappiness

A higher value means the kernel prefers swapping out anonymous memory over dropping page cache. A lower value means the kernel prefers keeping anonymous memory in RAM and drops page cache first. On a database server with a large page cache, lowering swappiness to 10 prevents the kernel from swapping out database memory in favor of cached files. On a system that runs many short-lived processes, a higher value keeps the file cache hot.

Set it temporarily:

sysctl vm.swappiness=10

Set it permanently:

echo "vm.swappiness=10" >> /etc/sysctl.conf

Watch Swap Activity

vmstat shows swap activity in real time. The si column shows pages swapped in from disk per second. so shows pages swapped out:

vmstat 1

On this VPS with swap disabled, both columns are zero. With swap enabled and memory pressure, you would see numbers in those columns. Non-zero so means the kernel is moving anonymous memory to disk. Non-zero si means it is moving it back.

Swap usage alone does not indicate a problem. If the kernel swaps out a process that has been idle for an hour to make room for a file cache that is actively used, that is normal behavior. High so combined with high CPU wait time (wa column) means the system is thrashing: constantly swapping instead of doing useful work. That is the problem.

When the OOM Killer Fires

When the kernel cannot free enough memory even after hitting the min watermark, the OOM killer selects a process to terminate. It scores every process based on memory usage, runtime, and other factors. The process with the worst score dies.

After an OOM kill, the kernel logs a message to dmesg:

dmesg | grep -i "out of memory" | tail -5
dmesg | grep -i killed | tail -5

Each OOM event records the process name, PID, and the amount of memory it was using. If a Java or Node process gets killed repeatedly, the application is trying to use more memory than the kernel is willing to give it. The fix is either to limit the process with cgroups or to give the system more RAM.

Enabling Swap

If your workload benefits from having swap, enable it. A 1 GB swap file:

fallocate -l 1G /swapfile
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo "/swapfile none swap sw 0 0" >> /etc/fstab

Swap is not a substitute for RAM. A system with 8 GB of RAM and 8 GB of swap running a workload that uses 12 GB will spend most of its time swapping instead of doing useful work. Swap exists to absorb memory spikes, to allow the kernel to move idle processes out of RAM, and to provide a safety net when a process unexpectedly tries to allocate more than is available.