Troubleshooting Node.js 'EMFILE: too many open files' on High-Traffic PM2 Servers

When running high-traffic Node.js applications using process managers like PM2, one of the most frustrating bottlenecks you can encounter is the dreaded EMFILE: too many open files error. This error often surfaces silently, bringing your production environment to its knees during traffic spikes.

In this guide, we’ll dive deep into what causes the EMFILE error in Node.js, how file descriptors work in Linux, and the definitive steps to resolve this limitation at the kernel and process levels.

Understanding the EMFILE Error

In Unix-like operating systems, everything is a file. This includes regular files, directories, network sockets, and pipes. When a Node.js application accepts an incoming HTTP request or connects to a database, the operating system assigns it a “file descriptor” (FD)—an integer used to track the open file or connection.

When your application attempts to open more files or sockets than the operating system permits for its user or process, the OS rejects the request, and Node.js throws the EMFILE error:

Error: EMFILE: too many open files, open '/path/to/some/file.txt'
    at Object.openSync (fs.js:476:3)
    at Object.readFileSync (fs.js:377:35)

For heavy web applications, this usually happens not because you are reading millions of text files, but because every active TCP connection consumes a file descriptor.

Diagnosing File Descriptor Limits

Before blindly increasing limits, it’s crucial to understand your current constraints.

1. Check Global File Descriptor Limits

First, check the maximum number of file descriptors the Linux kernel will allocate globally:

cat /proc/sys/fs/file-max

This number is usually quite high (e.g., in the millions). You can also check how many are currently in use:

cat /proc/sys/fs/file-nr

The output will look something like 1280 0 3253759. The first column is the number of allocated file descriptors, the second is the number of free allocated descriptors, and the third is the maximum.

2. Check Process-Specific Limits

The most common culprit is the soft and hard limits set for the specific user running the Node.js/PM2 process.

Check the current user’s limits using ulimit:

ulimit -n

By default, many Linux distributions restrict this to 1024. If your PM2 process handles more than 1024 concurrent connections/files, it will crash.

To see the limits specifically applied to a running PM2 process, find the PID of your Node application:

ps aux | grep node

Then, inspect its limits directly via the proc filesystem:

cat /proc/<PID>/limits | grep "Max open files"

The Solution: Tuning System and User Limits

To resolve the EMFILE error, you must increase the maximum open file limit for the user executing the PM2 process.

Step 1: Update system-wide limits (sysctl)

If your global limit /proc/sys/fs/file-max is too low, edit /etc/sysctl.conf:

fs.file-max = 2097152

Apply the changes:

sysctl -p

Step 2: Update User Limits (limits.conf)

Next, update the security limits for the user running PM2 (e.g., nodejs_user). Edit the /etc/security/limits.conf file:

nodejs_user    soft    nofile    65536
nodejs_user    hard    nofile    1048576

Additionally, ensure that the PAM module loads these limits. Check /etc/pam.d/common-session and /etc/pam.d/common-session-noninteractive to ensure this line exists:

session required pam_limits.so

Step 3: Handle systemd Contexts (Crucial for PM2)

If PM2 is set to start on boot via pm2 startup (which uses systemd), the limits in /etc/security/limits.conf might be ignored! systemd manages its own limits.

To fix this, you must override the systemd service file for PM2.

Find your PM2 service file (usually /etc/systemd/system/pm2-nodejs_user.service), and open it in an editor. Alternatively, use systemctl override:

sudo systemctl edit pm2-nodejs_user.service

Add the following under the [Service] block:

[Service]
LimitNOFILE=1048576
LimitNOFILESoft=65536

Reload the systemd daemon and restart the PM2 service:

sudo systemctl daemon-reload
sudo systemctl restart pm2-nodejs_user.service

Step 4: Verify the Fix

Restart your Node.js application within PM2:

pm2 restart all

Grab the new PID and check the limits again:

cat /proc/<PID>/limits | grep "Max open files"

You should now see the Soft Limit as 65536 and Hard Limit as 1048576.

Knowing When to Scale Up

While increasing the ulimit allows a single server to handle vastly more concurrent connections, eventually, you will run into hardware constraints—CPU context switching overhead, RAM limits, or network interface saturation.

If your web applications are outgrowing shared hosting or standard VPS environments and require deep kernel-level tuning for massive concurrency, it’s time to consider bare-metal performance. You can completely bypass shared OS file descriptor limits (ulimit) and virtualization overhead by migrating to highly tunable Dedicated Servers. For optimal latency and routing for businesses operating in South Asia, deploying on Dedicated Servers in Pakistan ensures that your high-traffic Node.js applications deliver the fastest possible response times without arbitrary resource capping.

By understanding how Linux handles file descriptors and ensuring your system is configured to support the scale of your application, you can eliminate EMFILE errors and keep your production PM2 environments stable under heavy load.