Innovate | Connect | Transform
ContactService statusUSD $

GICT knowledge base

How to Fix “Too Many Open Files” Error on Linux Servers

Published

LinuxLinux and CommandsServer Management

The “Too Many Open Files” error is one of those Linux issues that always seems to hit at the worst possible time. Your application has been running fine for weeks, traffic is up, things are going great, and then suddenly the server starts throwing errors and connections fail. The error message is technically accurate but completely unhelpful for figuring out what to do next. Most people end up googling the same set of ulimit commands and hoping for the best.

In this article, you will learn what file descriptors actually are, why Linux limits them, where the limits come from, how to find which process is hitting them, and how to raise the limits properly so the fix sticks across reboots and service restarts. By the end, you will know exactly how to diagnose and resolve this error instead of just copying commands from Stack Overflow.

Prerequisites

Before you start, make sure you have:

  • A Linux machine running Ubuntu 22.04, Debian 12, or a similar distribution.
  • A user account with sudo access.
  • Some familiarity with the command line.
  • A service or application that is hitting the file descriptor limit, or a willingness to test on a non-production system.

What File Descriptors Actually Are

The error mentions “open files,” but the term is misleading. In Linux, almost everything is treated as a file. Regular files on disk are files. Network sockets are files. Pipes are files. Devices are files. When a process opens any of these, the kernel gives it a small integer called a file descriptor that the process uses to refer to the resource.

A web server handling 5,000 concurrent connections has at least 5,000 file descriptors open just for those connections. Add log files, configuration files, database connections, and internal pipes, and the number gets higher. A busy server can easily have tens of thousands of open file descriptors at any moment.

The kernel tracks every open file descriptor and uses memory to do it. To prevent runaway processes from exhausting system resources, Linux puts limits on how many file descriptors each process and each user can have open at once. These limits are usually generous enough for normal workloads but can be too low for high-traffic servers.

When a process tries to open a new file descriptor and hits its limit, the underlying system call fails with EMFILE (too many open files for the process) or ENFILE (too many open files system-wide). The application sees this as an error and typically logs the famous “Too Many Open Files” message.

The Different Limit Layers

This is the part that confuses people. There is not one limit. There are several, and they stack on top of each other. Raising one without the others does not help.

System-Wide Limit

The kernel has a global limit on the total number of file descriptors that can be open across all processes. This is set by the fs.file-max kernel parameter and is usually high enough that you will not hit it on a normal server. On modern systems, this is often in the millions.

Per-Process Hard and Soft Limits

Each process has two limits: a soft limit and a hard limit. The soft limit is the actual ceiling that applies to the process. The hard limit is the maximum the soft limit can be raised to without root privileges.

A process can lower its own limits or raise the soft limit up to the hard limit. Only root can raise the hard limit. These limits come from the parent process at fork time, which is why they often need to be configured for the user or service that runs the application, not just at the global level.

User Limits

Linux can apply limits per user, configured in /etc/security/limits.conf or files under /etc/security/limits.d/. These limits apply when the user logs in or when a process is run under that user, but only through certain mechanisms (specifically PAM-aware ones).

Systemd Service Limits

If your service runs under systemd, which is the case for most modern Linux services, systemd applies its own limits and ignores /etc/security/limits.conf entirely. This is the single most common reason people raise limits and find that nothing changes. They edited the wrong file.

The systemd limits come from the unit file or from drop-in configuration files. If you want to raise limits for a systemd service, you need to do it in the systemd configuration, not through the traditional Linux limits files.

Diagnose the Problem First

Before changing anything, figure out what is actually hitting the limit. Sometimes the right answer is not “raise the limit” but “find the bug that is leaking file descriptors.”

Check the Current Limits

See the soft and hard limits for your current shell.

ulimit -Sn
ulimit -Hn

The first command shows the soft limit, the second shows the hard limit. The default on most distributions is 1024 soft and 4096 hard, which is too low for many server workloads.

Check the system-wide maximum.

cat /proc/sys/fs/file-max

This is usually a large number and rarely the actual problem.

See how many file descriptors are currently in use system-wide.

cat /proc/sys/fs/file-nr

The output shows three numbers: allocated file descriptors, free allocated file descriptors, and the maximum. If the first number is close to the third, the system is genuinely running out.

Check the Limits of a Running Process

Find the process ID of your application.

pgrep -f nginx

Replace nginx with the name of your service.

Check the limits for that specific process.

cat /proc/12345/limits

Replace 12345 with the actual process ID. Look for the line about “Max open files.” This shows the actual limit applied to the running process, which is what matters.

Count how many file descriptors the process currently has open.

ls /proc/12345/fd | wc -l

If this number is close to the soft limit shown in the previous command, the process is about to hit the limit or has already hit it.

Find What the Process Has Open

If the count is suspiciously high, see what the process is actually keeping open.

sudo ls -la /proc/12345/fd | head -50

The output shows symlinks pointing to whatever each file descriptor refers to. Lots of network sockets are normal for a web server. Lots of references to deleted files might indicate a log rotation problem. Thousands of references to the same database might indicate a connection leak in the application code.

Raise the Limits the Right Way

Once you have confirmed that you genuinely need more file descriptors and not just to fix a leak, the next step is raising the limits in a way that actually applies to your service.

For Systemd Services

This is the right approach for almost anything running on modern Linux. The fix is per-service and survives restarts properly.

Create a systemd drop-in directory for your service. The example uses Nginx, but the same approach works for any service.

sudo mkdir -p /etc/systemd/system/nginx.service.d

Create a configuration file inside that directory.

sudo nano /etc/systemd/system/nginx.service.d/limits.conf

Add the following content.

[Service]
LimitNOFILE=65535

The value 65535 is reasonable for most workloads. For very large servers, you can go higher. Setting it to infinity removes the limit entirely, but that is rarely a good idea.

Reload systemd so it picks up the new configuration.

sudo systemctl daemon-reload

Restart the service.

sudo systemctl restart nginx

Verify the new limit is applied.

cat /proc/$(pgrep -f nginx | head -1)/limits | grep "Max open files"

The number should match what you set. If it does not, the change has not been picked up correctly.

For Login Shells and Non-Systemd Processes

If you have processes running outside of systemd, like applications launched from a user’s shell or via cron, you need the traditional limits configuration.

Edit the limits configuration.

sudo nano /etc/security/limits.conf

Add lines for the relevant user. The example below sets limits for the appuser user.

appuser soft nofile 65535
appuser hard nofile 65535

The first column is the user, the second is the type of limit, the third is the resource (nofile for number of open files), and the fourth is the value. To apply the limit to all users, use * as the user.

Make sure PAM is configured to apply these limits. Check that pam_limits.so is included in the relevant PAM configuration files.

grep pam_limits /etc/pam.d/*

If it is not present in /etc/pam.d/common-session, add it.

session required pam_limits.so

The user needs to log out and back in for the new limits to take effect. New shells will see the new limits, but existing ones will not.

Raise the System-Wide Limit If Necessary

Most servers do not need this, but if you are running a system with hundreds of thousands of connections across many processes, the system-wide limit can become the bottleneck.

Check the current value.

cat /proc/sys/fs/file-max

Set a higher value temporarily.

sudo sysctl -w fs.file-max=2097152

Make the change permanent by adding it to /etc/sysctl.conf or a file in /etc/sysctl.d/.

sudo nano /etc/sysctl.d/99-file-max.conf

Add the following line.

fs.file-max = 2097152

Apply the change.

sudo sysctl -p /etc/sysctl.d/99-file-max.conf

[!NOTE] The system-wide limit affects the entire machine, not individual processes or users. Each process is still bound by its own per-process limit. Raising the system-wide limit without also raising per-process limits does not help individual processes.

Common Service-Specific Configurations

Some services have their own limits that interact with the system limits. Knowing about these saves a lot of debugging time.

Nginx

Nginx has its own worker_rlimit_nofile directive that sets the limit for worker processes. This needs to be set in addition to the systemd limit, and it cannot exceed the system limit.

Edit the main Nginx configuration.

sudo nano /etc/nginx/nginx.conf

Add or update the directive at the top level, outside of any block.

worker_rlimit_nofile 65535;

events {
    worker_connections 16384;
}

Note that worker_connections is also relevant. The maximum number of concurrent connections Nginx can handle is roughly worker_connections multiplied by the number of worker processes.

Test and reload.

sudo nginx -t
sudo systemctl reload nginx

MySQL and MariaDB

MySQL has its own limit configured through open_files_limit. By default, MySQL tries to set this to a high value, but it cannot exceed the operating system limit set by systemd.

Edit the MySQL configuration.

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Add or update the directive in the [mysqld] section.

[mysqld]
open_files_limit = 65535

Make sure the systemd unit allows it. Create or edit a drop-in.

sudo systemctl edit mysql

Add the following content.

[Service]
LimitNOFILE=65535

Restart MySQL.

sudo systemctl daemon-reload
sudo systemctl restart mysql

Docker

Docker containers inherit limits from the Docker daemon, which has its own configuration. If a container hits file descriptor limits, the fix is usually in the daemon configuration.

Edit the Docker daemon configuration.

sudo nano /etc/docker/daemon.json

Add or update the default-ulimits section.

{
    "default-ulimits": {
        "nofile": {
            "Name": "nofile",
            "Hard": 65535,
            "Soft": 65535
        }
    }
}

Restart Docker.

sudo systemctl restart docker

You can also override limits per container using the --ulimit flag when starting a container.

When the Real Problem Is a Leak

If your file descriptor count grows steadily over time and never comes down, you have a leak in the application, not a configuration issue. Raising the limit just delays the inevitable.

Common causes of file descriptor leaks:

  • Network connections that never get closed. Code that creates connections to databases, caches, or external services and forgets to close them in some error path. Connection pools that have bugs and lose connections instead of returning them to the pool.
  • File handles in long-running scripts. Background workers that open files and never close them, especially in languages where garbage collection is not predictable.
  • Goroutines or threads that leak. Spawning new workers on every request without proper cleanup, where each worker holds connections.
  • Log file handles after rotation. Some applications keep writing to the old log file after logrotate runs, holding the deleted inode open. The fix is usually to send a SIGHUP to the process or use copytruncate in the rotation config.

To diagnose a leak, watch the file descriptor count over time.

watch "ls /proc/$(pgrep -f myapp | head -1)/fd | wc -l"

If the number grows steadily under load and never decreases, you have a leak. Looking at what file descriptors are open, using ls -la /proc/PID/fd, usually points to the problem.

[!WARNING] Raising file descriptor limits without understanding why you are hitting them can hide real bugs. If your service was running fine yesterday and is hitting limits today, something probably changed. Investigate before raising the ceiling.

Conclusion

You now understand what file descriptors are, why Linux limits them, the different layers where limits apply, and how to diagnose which one is biting you. You also know how to raise limits properly for systemd services, login shells, and specific applications like Nginx, MySQL, and Docker. And you know how to recognize when the right fix is to find a leak instead of raising the limit.

The “Too Many Open Files” error is annoying because the message is so generic, but the underlying issue is almost always one of two things. Either your service genuinely needs more headroom, in which case the systemd drop-in fix takes about two minutes. Or you have a leak, in which case raising the limit is just postponing a real problem. Knowing the difference is what separates a quick fix from another midnight pager.

Related guides

Account services are unavailable until the production backend is configured.