AlgoMaster Logo

Resource Limits

36 min readUpdated August 7, 2026
Listen to this chapter
Unlock Audio

A service can be correctly isolated from other users and still make the machine unusable.

A file-descriptor leak can exhaust the process's descriptor table. A runaway worker can create thousands of threads. A computation can consume CPU indefinitely, and an unexpected core dump can fill a filesystem.

Resource limits place ceilings on selected kinds of process activity. They protect availability by turning unbounded growth into a defined failure.

On Unix-like systems, the traditional interface is a family of limits commonly called rlimits. Linux exposes them through getrlimit(), setrlimit(), and prlimit().

A resource limit is a ceiling, not a reservation. It constrains what a process may consume but does not guarantee that the resource will be available.

Choosing the value is only half the design. The application must also handle the resulting error or signal safely.

Soft and Hard Resource Limits

The C interface represents one resource limit as:

The soft limit is the value currently enforced by the kernel.

The hard limit is the ceiling to which an unprivileged process may raise its soft limit.

Suppose a process has:

It can lower its soft limit or raise it as high as 65,536. It cannot raise the hard limit without appropriate privilege.

An unprivileged process may also lower its hard limit:

That hard-limit reduction is irreversible for the process and its descendants unless they later execute with authority that permits raising it.

The special value RLIM_INFINITY means that this rlimit imposes no ceiling. It does not mean the underlying resource is infinite; physical capacity and other system limits still apply.

The soft limit cannot exceed the hard limit:

An attempt to violate that relationship fails.

Resource Limits as Process State

Resource limits are attributes of a process and are shared by its threads.

A child created with fork() inherits copies of its parent's limits. The limits remain in effect across execve():

The shell or service manager establishes the limits, the child process is created, and after execve() the application starts with those inherited limits.

This is why ulimit is implemented as a shell built-in. An external command could change only its own limits and those of its future children; it could not reach backward and change the shell that launched it.

Changing a terminal's limit affects commands launched from that shell afterward. It does not change:

  • Processes that are already running
  • Services started by a separate service manager
  • Other login sessions

Most rlimits are enforced independently for each process. If a process forks, each child receives its own descriptor limit, address-space limit, and CPU-time accounting.

That per-process scope is useful but incomplete for a multi-process service. A tree of 100 workers can collectively consume much more than any one worker's ceiling.

Some limits use different accounting scopes. RLIMIT_NPROC, for example, counts Linux threads belonging to a real UID rather than only descendants of one process.

Inspecting Limits Before Changing Them

For the current Bash shell, display all limits with:

Inspect the soft and hard open-file limits separately:

Linux exposes a process's limits in:

Representative rows look like:

The actual values depend on the distribution, login configuration, service manager, and process ancestry.

The prlimit utility provides another view:

It can also start a command with selected limits:

Here 1024 is the soft limit and 4096 is the hard limit.

Inspect the running application rather than relying on a configuration file or the administrator's shell. Inheritance paths often explain why the effective value differs from the intended one.

The Main Linux Resource Limits

The most operationally important rlimits are:

LimitWhat it boundsTypical limit result
RLIMIT_NOFILEFile-descriptor numbers available to one processEMFILE
RLIMIT_NPROCLinux threads belonging to one real UIDfork() or thread creation fails with EAGAIN
RLIMIT_ASTotal virtual address-space sizemmap() or brk() fails with ENOMEM
RLIMIT_DATAData segment, heap, and some mappingsAllocation path fails with ENOMEM
RLIMIT_STACKProcess stack sizeSIGSEGV on failed stack growth
RLIMIT_CPUConsumed CPU timeSIGXCPU, then SIGKILL at the hard limit
RLIMIT_FSIZEMaximum size of a created fileSIGXFSZ or EFBIG
RLIMIT_COREMaximum core-dump sizeDump omitted or truncated
RLIMIT_MEMLOCKMemory an unprivileged process may lock in RAMLocking operation fails

Other Linux limits cover queued signals, POSIX message queues, real-time priority, and continuous real-time CPU use.

The failure is part of the interface. A limit that causes an application to retry in a tight loop can make availability worse rather than better.

Loading simulation...

RLIMIT_NOFILE: Open File Descriptors

RLIMIT_NOFILE is the resource limit backend services encounter most often.

Despite its name, it is not limited to regular files. File descriptors represent:

  • Client and server sockets
  • Pipes
  • Event-notification objects
  • Open directories
  • Device handles
  • Log files
  • Database files
  • Many runtime and monitoring interfaces

The limit is one greater than the highest descriptor number the process may allocate.

With a soft limit of 1024, newly allocated descriptors must have numbers below 1024. Existing descriptors are not closed if the limit is lowered beneath their numbers, but further allocation is constrained.

Operations such as open(), socket(), pipe(), accept(), and dup() fail with:

EMFILE means the calling process hit its per-process descriptor limit.

A different error means the system-wide open-file table was exhausted:

The global ceiling is separate from RLIMIT_NOFILE. Raising one service's rlimit does not raise the kernel-wide limit.

Linux exposes two related system ceilings:

nr_open caps how high a process's RLIMIT_NOFILE hard limit may be raised. file-max controls the system-wide open-file table associated with ENFILE.

On current Linux, RLIMIT_NOFILE also limits how many descriptors an unprivileged process may have in flight while passing them to other processes over Unix-domain sockets.

Budget Descriptors from Concurrency

A network service should derive its descriptor requirement from its workload rather than copy an arbitrary large value.

A useful estimate is:

Suppose one service instance can hold:

Its expected requirement is roughly 14,200, so a soft limit of 1,024 is clearly too small.

The estimate must reflect peak simultaneous use, not total opens over time. A descriptor that is closed can be reused.

Count a running process's open descriptors with:

The count is a snapshot and can change while it is being measured. Tools may also need permission to inspect another process.

A limit should sit above legitimate peaks while remaining low enough to contain a leak or compromised process.

Descriptor Limits vs. Descriptor Leaks

Suppose a service leaks one descriptor per request.

Raising its limit from 1,024 to 65,536 delays the failure:

LimitEffect on the same leak rate
1,024Fails sooner
65,536Fails later, and consumes more kernel state first

The larger limit may be necessary for expected concurrency, but it does not repair incorrect lifecycle management.

Monitor both:

The ratio helps distinguish a naturally high-concurrency service from one approaching its ceiling unexpectedly.

When a server's accept() fails with EMFILE, immediately retrying can create a busy loop while clients continue to arrive. Robust servers preserve enough control to log, shed load, and recover descriptors rather than spinning.

One established technique keeps an emergency descriptor open. On EMFILE, the server closes that descriptor, accepts and closes one pending connection, then reopens the emergency descriptor. This does not fix the exhaustion; it gives the process enough room to reject work cleanly while the underlying leak or overload is handled.

The Practical Descriptor Ceiling of select()

Raising RLIMIT_NOFILE does not guarantee that every I/O API can represent the resulting descriptor numbers.

On Linux, the traditional select() interface uses a fixed-size fd_set that normally supports descriptor numbers only below FD_SETSIZE, commonly 1024.

A process with:

can receive descriptor 4,000, but passing that descriptor to an unmodified select()-based implementation is unsafe or unsupported.

High-concurrency Linux services normally use interfaces designed for large descriptor sets, such as epoll, or a runtime that uses an appropriate mechanism internally.

The operational lesson is:

Raise the limit only after confirming that the application and its libraries can handle descriptor numbers above the historical select() ceiling.

RLIMIT_NPROC: Processes and Threads per Real UID

RLIMIT_NPROC has a misleading name on Linux. It limits the number of existing threads associated with the calling process's real UID.

When the UID's count has reached the soft limit:

Thread creation can also fail with EAGAIN for other reasons, so the errno alone does not prove that RLIMIT_NPROC was the active ceiling.

The limit is not scoped to one service tree. If several services run under the same real UID, their threads contribute to the same account-wide count:

The limit is per user, not per service. One service exhausting the pool prevents the others from creating threads, even though they are unrelated.

One workload can therefore prevent another workload under the same UID from creating a thread.

Linux does not enforce this limit for a process whose real UID is 0 or that holds particular administrative resource capabilities. Running a service as root defeats this protection.

RLIMIT_NPROC is useful for containing one service account, but it is not an exact “number of children this parent may create” setting.

RLIMIT_AS: Virtual Address Space, Not RAM

RLIMIT_AS limits the total size of a process's virtual address space.

The kernel applies it to operations such as:

Allocation or mapping growth that would exceed the limit normally fails with ENOMEM. Failed automatic stack expansion can produce SIGSEGV.

This limit does not directly mean:

A mapped region counts toward virtual address space even when few of its pages are resident. Conversely, an allocation can reuse space already mapped by an allocator without immediately increasing the address-space total.

Many managed runtimes reserve large virtual ranges for heaps, compressed pointers, garbage collectors, or address-space organization. A low RLIMIT_AS can prevent them from starting even when the machine has ample free RAM.

Use RLIMIT_AS when bounding virtual mappings is the intended policy. Do not present it as an exact physical-memory budget.

The Narrow, Runtime-Dependent Scope of RLIMIT_DATA

RLIMIT_DATA traditionally limits the process's initialized data, uninitialized data, and heap.

It affects brk() and sbrk() growth. On modern Linux it also affects certain mmap() allocation paths.

Applications do not all acquire dynamic memory in the same way. A C allocator may use both the traditional heap and anonymous mappings. A language runtime may reserve its own large regions.

This makes RLIMIT_DATA less intuitive as a general service memory ceiling. The same configured value can affect different runtimes in different ways.

When it is exceeded, allocation paths normally fail with ENOMEM. Application code must handle allocation failure rather than assuming that memory allocation cannot return an error.

The Limits of RLIMIT_RSS on Modern Linux

RLIMIT_RSS is named as though it limits resident physical memory.

Modern Linux does not implement it as a general resident-set ceiling. Setting it has no useful effect on ordinary process memory consumption.

This is an important operational trap:

Do not use ulimit -m or systemd's LimitRSS= expecting it to contain a service's physical-memory usage on current Linux.

A service-wide physical-memory policy requires a group-aware mechanism that accounts for the processes together. That is a different model from a traditional per-process rlimit.

RLIMIT_STACK: Stack Growth

RLIMIT_STACK specifies the maximum stack size for a process.

When automatic stack growth reaches the limit, the access generates SIGSEGV. A process that intends to handle this condition needs an alternate signal stack because its ordinary stack may no longer be usable.

A very small value can break:

  • Deep call chains
  • Large local variables
  • Recursive algorithms
  • Runtime startup

On Linux, the stack limit also constrains the combined space available for command-line arguments and environment strings during execve().

Thread runtimes often use the inherited stack limit when choosing default thread-stack sizes, but separately created thread stacks are mappings with runtime-controlled sizes. RLIMIT_STACK should not be interpreted as the sum of every thread's stack allocation.

The safer application design avoids unexpectedly large stack use instead of relying only on a high limit.

RLIMIT_CPU: Consumed CPU Time

RLIMIT_CPU measures CPU time consumed by the process, in seconds. It is not a wall-clock timeout.

A process that sleeps for an hour while consuming almost no CPU does not spend an hour of this budget.

At the soft limit, Linux sends:

The default action terminates the process. A program can catch the signal and begin an orderly shutdown.

If it continues consuming CPU, Linux sends SIGXCPU again periodically. At the hard limit, the kernel sends:

which cannot be caught.

This limit is cumulative over the process lifetime. It is often a poor way to express “use no more than 50% of one CPU” for a healthy long-running service, because the service eventually consumes the fixed total even at a modest rate.

It is better suited to bounded computations whose total CPU work should not exceed a known budget.

RLIMIT_RTTIME is a separate Linux limit for a process under real-time scheduling. It bounds continuous CPU time consumed without making a blocking system call and helps contain a runaway real-time task.

RLIMIT_FSIZE: Maximum File Growth

RLIMIT_FSIZE limits how large a file created or extended by the process may become.

When an operation would exceed the soft limit, the process receives:

The default action terminates the process. If the signal is handled or ignored, operations such as write() or truncate() fail with:

This limit can contain a runaway file writer, but it is not a disk-space quota. It limits the size of an individual file produced by the process, not the sum of all files the identity owns.

A logging service must handle EFBIG without corrupting state or retrying forever. Rotation and retention policy remain necessary even when a file-size limit exists.

RLIMIT_CORE: Core-Dump Size and Secret Exposure

A core dump captures process memory and execution state after certain crashes. It can be invaluable for debugging, but it may contain:

  • Credentials and tokens
  • Customer data
  • Cryptographic material
  • In-memory request contents

RLIMIT_CORE specifies the maximum core-file size. A soft limit of 0 normally prevents a conventional core file:

A nonzero limit can truncate a larger dump, which may make the result incomplete.

Core-dump creation also depends on filesystem permissions, dumpability rules, kernel configuration, and the system's crash handler. In particular, a system configured to pipe dumps to a handler can treat RLIMIT_CORE differently.

The policy should balance incident diagnosis against storage consumption and confidentiality. Core files need protection appropriate to the memory they contain.

RLIMIT_MEMLOCK: Memory Pinned in RAM

Applications can ask the kernel to lock selected pages in RAM with interfaces such as mlock() and mlockall().

Locked memory is useful when paging would be unacceptable, but it reduces the memory the kernel can reclaim. Allowing every process to lock unlimited memory would threaten system availability.

RLIMIT_MEMLOCK caps how many bytes an unprivileged process may lock. The effective value is rounded to page granularity.

Exceeding the limit makes the locking operation fail. It does not automatically terminate the process or reduce the mapping's ordinary address-space size.

Applications that genuinely need locked buffers should:

Locking a page also does not change which processes can read it. Memory access protection and memory residency are separate properties.

Limits for Queues and Scheduling Authority

Linux provides several less frequently encountered rlimits.

RLIMIT_SIGPENDING bounds queued signals for the calling process's real UID. The accounting is shared across the UID rather than isolated to one process.

RLIMIT_MSGQUEUE bounds kernel memory allocated for POSIX message queues owned by a real UID. The accounting includes both payload and kernel bookkeeping.

RLIMIT_RTPRIO places a ceiling on the real-time priority a process may request.

RLIMIT_NICE bounds how far a process may improve its scheduling priority through nice values. Its numeric resource-limit representation does not directly match the familiar -20 through 19 nice scale.

These limits constrain access to globally important kernel resources and scheduling authority. As with RLIMIT_NPROC, their accounting scope may be the real UID rather than a single service.

Lowering a Limit Below Current Use

Linux generally allows a process to lower a soft limit below its current consumption.

Suppose a process already has 200 open descriptors and lowers RLIMIT_NOFILE to 100:

The kernel does not close 100 descriptors automatically.

This behavior is useful for dropping future authority after startup, but it means that lowering a limit is not retroactive cleanup.

The same principle appears with other resources: a limit commonly prevents further growth or causes the next relevant operation to fail. It does not necessarily reclaim what the process already acquired.

Before lowering a live service's limit, understand both current use and failure behavior. The next routine allocation can fail immediately.

Hands-On: Triggering EMFILE

The following Linux program lowers its soft RLIMIT_NOFILE, repeatedly opens /dev/null, and confirms that descriptor allocation eventually fails with EMFILE.

It lowers only the soft limit, leaving the original hard ceiling unchanged:

Compile and run:

The number of successfully opened descriptors is less than the soft limit because the process already has descriptors such as standard input, output, and error.

The exact count can vary with the execution environment. The important result is:

The program's limit disappears when the process exits. Its parent shell remains unchanged because the child modified only its own process state.

Configuring Limits for a systemd Service

A systemd service can set rlimits in its unit:

A single value sets both the soft and hard limit. Different values use soft:hard syntax:

After updating a unit, reload the service-manager configuration and restart the process. Then verify the actual runtime values:

The service manager can establish values independently of an administrator's interactive ulimit. Changing the shell limit does not affect a service already launched by PID 1.

LimitNPROC= retains the real-UID-wide semantics of RLIMIT_NPROC. It does not mean “this unit may have exactly this many tasks,” especially when several services share the same UID.

Likewise, LimitRSS= does not become functional merely because it appears in a unit. Modern Linux still does not enforce RLIMIT_RSS.

Resource Limits vs. Group Resource Controls

Traditional rlimits are attached to a process and inherited by descendants. Most are then accounted per process.

That creates an escape from aggregate accounting:

The limit applies per process rather than to the group. Three children can collectively exceed what any one of them could reach alone.

A group-level resource controller instead accounts for a collection of processes together. It can express policies such as a total memory budget or a maximum number of tasks for an entire service.

The models solve different problems:

Use an rlimit when its specific failure semantics and scope match the requirement. Use aggregate accounting when the service as a whole must stay within a budget.

They can be combined. A service may have a group memory budget while each worker also has a descriptor ceiling and core-dump policy.

Resource Limits vs. Capacity Guarantees

A process with RLIMIT_NOFILE=65,536 is allowed to allocate descriptor numbers in that range. It is not guaranteed that the system-wide file table, memory, or network capacity can satisfy all of them.

Likewise:

  • An unlimited address-space rlimit does not guarantee successful allocation.
  • A high process limit does not guarantee that thread stacks can be allocated.
  • A large file-size limit does not guarantee free disk space.
  • A permitted locked-memory budget does not guarantee that every lock request succeeds.

Limits answer:

Capacity planning answers:

A production design needs both.

Choosing Limits from Failure Budgets

Setting every resource to unlimited maximizes flexibility but removes containment.

Setting every limit aggressively low creates self-inflicted outages.

A practical limit should be based on:

  1. Measured normal and peak use
  2. Expected concurrency and workload growth
  3. Runtime and library overhead
  4. Headroom for deployment and failure paths
  5. The damage acceptable if the process is compromised
  6. The application's behavior when the limit is reached

Test the failure intentionally. A service that handles EMFILE cleanly in design documents but has never encountered it under test should not be assumed resilient.

Alerts should fire before the hard boundary. Once the resource is fully exhausted, the process may be unable to open a log, create a diagnostic pipe, allocate recovery memory, or start a helper.

Resource limits work best as the final containment boundary behind ordinary load shedding and application-level backpressure.

Diagnosing a Limit-Related Failure

Start with the failing operation and errno or signal:

Then inspect the affected process:

Compare the soft limit with current consumption using a resource-specific tool. A configured hard limit is not necessarily the active failure boundary; the soft value is what the kernel enforces.

Trace the process's ancestry and launcher configuration:

The current process value is the result of the login or service-manager defaults, then any unit or shell override, then any application self-adjustment.

Finally, distinguish a per-process limit from a system-wide shortage. EMFILE and ENFILE sound similar but identify different scopes. ENOMEM can describe an rlimit, an actual memory shortage, mapping constraints, or other allocation failures.

Do not raise a limit until the resource's current use and growth pattern are understood.

Summary

Resource limits place kernel-enforced ceilings on selected process resources. Each resource has an enforced soft limit and a hard ceiling that bounds unprivileged changes.

Limits are inherited across fork() and preserved across execve(). Threads in one process share them, while most accounting remains per process. Some limits, notably RLIMIT_NPROC, use real-UID-wide accounting.

RLIMIT_NOFILE controls descriptor allocation and produces EMFILE; it is distinct from system-wide ENFILE. Descriptor values must be sized from concurrency, monitored in production, and supported by the application's I/O APIs.

Address-space, stack, CPU-time, file-size, core-dump, and locked-memory limits each have different units and failure behavior. RLIMIT_AS is not a resident-memory budget, and RLIMIT_RSS is not effective on modern Linux.

Shells, service managers, and applications can establish limits. Always verify /proc/<pid>/limits on the running process and test how the application behaves at the boundary.

An rlimit is a ceiling rather than a reservation or whole-service budget. Effective availability protection combines correctly scoped limits, capacity planning, monitoring, and graceful failure handling.

Quiz

Resource Limits Quiz

5 quizzes