Innovate | Connect | Transform
ContactService statusUSD $

GICT knowledge base

Complete Guide: How TCP Keepalives Keep Long-Idle Connections Alive

Published

Linux Networking
Complete Guide: How TCP Keepalives Keep Long-Idle Connections Alive

You have an SSH tunnel running to a VPS behind a corporate NAT gateway. You leave it open while you go to a meeting. When you come back an hour later, the terminal is frozen. The connection is dead. You did not change anything, but something in the network between your laptop and the VPS decided the connection no longer exists.

Or you run a web service with a PostgreSQL connection pool. The pool was built at startup and has been working fine for days. At 3am, a request comes in and the pool serves a connection that was closed server-side six hours ago. The request fails with a strange timeout. The pool did not know the connection was dead.

Both problems have the same root cause. Network hardware that tracks TCP connection state has a timeout. When a connection goes idle long enough, the NAT gateway or firewall drops the state entry. Both ends keep transmitting data into a void, not knowing the path is gone. TCP keepalives are the fix.

What TCP Keepalive Actually Is

When a TCP connection is idle, neither side has any reason to send a packet. The network hardware along the path has no traffic to observe, so it cannot know the connection still exists. After enough time with no packets, state-tracking devices drop their connection records. When one endpoint finally does send data, it gets no response because the other side closed the connection long ago.

TCP keepalive solves this by sending a probe packet when a connection has been idle for a configurable period. The probe contains no data and has the ACK flag set. It is not part of the normal data exchange, but the TCP protocol allows it. If the remote endpoint is still alive, it responds with an ACK. If the response never comes after enough retries, the local kernel concludes the path is dead and closes the socket.

There is one detail that trips up most people. The kernel has keepalive support built in, but it does not apply it to any socket by default. Every application must explicitly request keepalive with the SO_KEEPALIVE socket option. Most programs do not do this. SSH does, PostgreSQL does, and some reverse proxies do. Many applications leave it off and rely entirely on the application layer to detect dead connections.

The Three Parameters

Linux exposes three kernel parameters that control keepalive behavior. You can read them from procfs or query them with sysctl.

sysctl net.ipv4.tcp_keepalive_time
sysctl net.ipv4.tcp_keepalive_intvl
sysctl net.ipv4.tcp_keepalive_probes

The default values on most Linux systems are:

tcp_keepalive_time = 7200      # seconds
tcp_keepalive_intvl = 75        # seconds
tcp_keepalive_probes = 9       # count

tcp_keepalive_time sets how long the connection must be idle before the first probe is sent. An idle connection means no data has been transmitted in either direction. Plain ACK packets with no payload do not count as data. Two hours is the default, which is far too long for most practical uses.

tcp_keepalive_intvl sets the interval between each subsequent probe. After the first probe is sent, the kernel keeps resending probes at this interval if no response arrives.

tcp_keepalive_probes sets how many unanswered probes the kernel will send before declaring the connection dead and notifying the application.

Here is the math on the default behavior. The connection sits idle for 7200 seconds (2 hours). The first probe goes out. No response. The kernel sends another probe every 75 seconds. After 9 unanswered probes, the socket is closed. That is 8 intervals of 75 seconds between probes, so the total time from first probe to closed socket is roughly 7200 + (9 times 75) = 7875 seconds, or about 2 hours and 11 minutes. The connection is not declared dead until more than two hours of inactivity plus 11 minutes of failed probes have passed.

Per-Socket Overrides

The three sysctl parameters set system-wide defaults. Individual applications can override them on a per-socket basis using socket options at the IPPROTO_TCP level. These three options correspond to the sysctl parameters:

TCP_KEEPIDLE   # overrides tcp_keepalive_time
TCP_KEEPINTVL  # overrides tcp_keepalive_intvl
TCP_KEEPCNT    # overrides tcp_keepalive_probes

Here is what this looks like in Python:

import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 60)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 10)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, 6)

This enables keepalive on the socket and overrides the system defaults to probe after 60 seconds of idle time, then retry every 10 seconds, and give up after 6 failed probes. The application layer has the final say over these values, which is why PostgreSQL, Redis, and other long-lived services expose their own keepalive tuning parameters.

Configuring the Sysctl

To change the system-wide defaults, use sysctl or write directly to procfs. Both do the same thing. To probe the current values without reading three separate files:

sysctl net.ipv4.tcp_keepalive_time net.ipv4.tcp_keepalive_intvl net.ipv4.tcp_keepalive_probes

To change the values immediately (they survive until the next reboot):

sysctl -w net.ipv4.tcp_keepalive_time=60
sysctl -w net.ipv4.tcp_keepalive_intvl=10
sysctl -w net.ipv4.tcp_keepalive_probes=6

Or write directly to procfs:

echo 60 > /proc/sys/net/ipv4/tcp_keepalive_time
echo 10 > /proc/sys/net/ipv4/tcp_keepalive_intvl
echo 6   > /proc/sys/net/ipv4/tcp_keepalive_probes

To make these changes persist across reboots, create a file in /etc/sysctl.d/:

net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6

Apply it without rebooting:

sysctl --system

Realistic values depend on your use case. For a web server with many short-lived connections, the defaults are fine. For an SSH tunnel behind NAT, you want probes much sooner. A reasonable SSH keepalive config looks like tcp_keepalive_time=60, tcp_keepalive_intvl=10, tcp_keepalive_probes=5, which detects a dead connection in about 60 seconds of inactivity plus 50 seconds of failed probes.

SSH Keepalives Are Not TCP Keepalives

SSH has its own keepalive mechanism that operates at the application layer. This is a critical distinction. When you set ServerAliveInterval in your SSH client config or on the command line, SSH sends a keepalive message through the encrypted SSH channel, not as a raw TCP packet.

ssh -o ServerAliveInterval=60 -o ServerAliveCountMax=3 [email protected]

The message is encrypted inside the SSH stream. A NAT gateway that only inspects TCP headers cannot see it and therefore cannot use it to refresh connection state. The SSH-level keepalive still works to detect a dead peer, but it does not solve the NAT timeout problem because it does not generate TCP-level traffic that the NAT device can observe.

The TCPKeepAlive directive in sshd_config controls something different. Setting TCPKeepAlive yes tells SSH to use the OS-level TCP keepalive mechanism, which does send real TCP packets. Setting TCPKeepAlive no uses only SSH-level keepalives. The default is yes.

The practical difference: if you are behind a NAT that times out idle connections, you need TCP-level keepalives (either TCPKeepAlive yes in sshd, or system-wide sysctl tuning). If you want to detect that the remote SSH process has crashed without caring about NAT, SSH-level keepalives suffice.

Real-World Application Settings

Most production databases and message queues expose keepalive parameters as configuration options. Here is how PostgreSQL handles this. The tcp_keepalives_idle parameter sets the TCP_KEEPIDLE value on the client socket that PostgreSQL opens. Similarly, tcp_keepalives_interval maps to TCP_KEEPINTVL, and tcp_keepalives_count maps to TCP_KEEPCNT.

# postgresql.conf
tcp_keepalives_idle = 60
tcp_keepalives_interval = 10
tcp_keepalives_count = 6

With these settings, PostgreSQL clients probing a connection that has been idle for 60 seconds will trigger TCP keepalive probes. If the OS does not receive an answer to 6 consecutive probes spaced 10 seconds apart, the socket is closed and PostgreSQL marks the connection as dead. Your connection pool can then retire the bad connection before it serves a stale socket to application code.

MySQL has similar settings. Redis uses tcp-keepalive in its redis.conf, expressed in seconds. HAProxy exposes keepalive tuning at the proxy level. Most tools that maintain long-lived TCP connections have some way to configure this, because the problem is universal.

When Keepalives Are Not Enough

TCP keepalives detect dead network paths. They do not detect dead application processes. If the remote machine is still reachable, the TCP stack will respond to keepalive probes with an ACK even if the application process has crashed and left its sockets in a half-open state. For this reason, most protocols that care about application liveliness implement their own heartbeat at the application layer, on top of TCP keepalives.

WebSocket connections are a good example. A WebSocket server that goes silent for 30 seconds may have crashed without closing the underlying TCP connection. A higher-level ping/pong mechanism at the WebSocket protocol layer detects this faster than any TCP keepalive configuration can. TLS session tickets that reestablish connections after long idle periods also interact with keepalive behavior in ways that pure TCP tuning cannot solve.

TCP keepalives are the right tool for keeping NAT and firewall state tables active. Application-level heartbeats are the right tool for detecting crashed processes. Using both together covers the real failure modes that production systems actually encounter.

Ready to tune your connections? InterServer VPS plans start at $6/month with full root access, so you can configure sysctl parameters, SSH settings, and database keepalives exactly as your workload requires.

Related guides

Account services are unavailable until the production backend is configured.