An application writes 128 KiB to a file with one system call. That does not mean the storage device receives one 128 KiB command.

The filesystem may map the file region to several device ranges. The device may impose a maximum request size. Adjacent work from another process may be combined with it. A scheduler may delay or reorder the resulting requests, and a multi-queue controller may execute several of them concurrently.

The kernel component that organizes this work is the block layer.

The block layer sits between higher-level kernel subsystems and block-device drivers. It accepts operations expressed in logical block ranges, adapts them to device limits, queues them, and dispatches them to drivers.

The block layer turns storage intentions into device-sized, schedulable requests.

Understanding this boundary explains why one application operation need not equal one device command, why requests can complete out of order, and how Linux feeds modern NVMe hardware without making every CPU contend on one queue.

A block device stores data in an addressable sequence of fixed-size logical blocks. The host can request block 1,000 without reading blocks 0 through 999 first.

HDDs and SSDs are common block devices. Partitions, logical volumes, and virtual disks can also expose a block-device interface even when their data ultimately passes through other storage layers.

A stream-oriented device behaves differently. A terminal or serial port presents a sequence of bytes rather than a persistent array of independently addressable locations. The block layer exists for devices with random-access block semantics.

Logical blocks

A logical block address, or LBA, identifies a block in the address space exposed by the device. A request needs at least:

If a device exposes 512-byte logical blocks, a 4 KiB operation spans eight logical blocks:

The request might therefore be described as:

The word block appears at several layers. A filesystem block, virtual-memory page, device logical block, and NAND erase block do not have to be the same size. The block layer works with the device-facing address range and the constraints reported for that device.

The block layer does not understand files

By the time ordinary file I/O reaches the block layer, a higher layer has already translated the relevant file region into storage ranges.

The block layer sees something like:

It does not see:

This separation is intentional. Filesystems understand names, directories, file offsets, and allocation. The block layer understands storage operations, logical addresses, queues, and device limits.

Linux represents block I/O in stages. A simplified path is:

The exact path can differ for stacked or virtual devices, but the responsibilities remain useful.

A bio describes block I/O

Linux uses a structure called a bio, short for block I/O, to describe an operation submitted to the block layer.

Conceptually, a bio carries:

The memory for one logical operation does not have to be physically contiguous. A bio can describe multiple memory segments that together supply or receive the requested bytes.

A bio represents what an upper layer wants done. It is not necessarily the command that a driver will submit to hardware.

A request is prepared for a driver

The block layer builds requests for the device driver. One request can contain work from one or more compatible bios.

This distinction is central:

Several adjacent bios may become one request. One bio may also be split into several requests if it exceeds a device limit.

There is therefore no one-to-one rule among system calls, bios, block-layer requests, and device commands.

Block devices report capabilities and limits to the kernel. Examples include:

The block layer uses these values to construct work the driver and device can accept.

Splitting a large operation

Suppose an upper layer submits a 1 MiB operation, but the device permits at most 256 KiB in one request.

The block layer can split the operation:

One application-sized operation becomes four device-sized ones, which is why a syscall count and a device operation count do not have to match.

The requests may complete independently. The upper layer observes the original operation as complete only after all required pieces have finished successfully.

Splitting can also be required when the memory description has too many segments or when a request crosses a boundary imposed by a device or lower storage layer.

Alignment

An operation must respect the device's addressability rules. A device exposing 4 KiB logical blocks cannot directly perform a request beginning halfway through one of those blocks.

Higher layers normally submit compatible operations, while the block layer carries and enforces the device limits exposed through the storage stack.

Alignment does not guarantee optimal performance. A request may be valid at the logical block size yet still be smaller than the device's preferred I/O size. Validity and efficiency are different concerns.

Splitting turns one operation into several requests. Merging does the opposite: it combines compatible adjacent work into a larger request.

Suppose two write bios target consecutive ranges:

If their operation attributes and device constraints are compatible, the block layer may form:

This is a back merge because the new range is appended to the end of the existing request.

If an existing request covers LBAs 108–115 and a new bio covers 100–107, adding the new range at the beginning is a front merge.

Why merging helps

Merging reduces the number of requests the kernel and device must process. That can lower:

A larger contiguous request can also transfer bytes more efficiently than many tiny commands.

When requests cannot merge

Logical adjacency is not enough. Requests may remain separate when:

The block layer must preserve correctness before pursuing efficiency.

Waiting briefly can reveal merge opportunities

If every bio is dispatched immediately, a neighboring bio arriving a moment later cannot be merged with it in the block layer.

Linux can temporarily accumulate work so several bios become visible together. This behavior is often called plugging. When the plug is released, the block layer can merge and dispatch the accumulated work.

Waiting creates a trade-off. It can reduce request count and improve throughput, but holding a latency-sensitive request for too long would be counterproductive. The batching opportunity is therefore deliberately limited.

When several requests are pending, the block layer does not always dispatch them in submission order.

Reordering can serve several purposes:

This is the role of an I/O scheduler.

Reordering is constrained by correctness. Some operations establish dependencies that higher layers require. The block layer and driver must preserve the ordering rules carried with those operations rather than treating every request as independent.

Even independent requests submitted in order can finish out of order:

The device may have several commands in flight, and one can finish sooner than another. Neither multi-queue dispatch nor modern device protocols promise that independent requests complete in submission order.

Higher layers must explicitly express dependencies when order matters. Merely submitting A before B is not a general completion-order guarantee.

Older block-layer designs used one main request queue protected by a shared lock.

That design was reasonable when an HDD's mechanical latency dominated the entire path. The kernel could spend time sorting requests while the disk head took milliseconds to move.

Fast SSDs changed the balance. Multiple CPU cores could submit hundreds of thousands of requests per second, and a single shared queue became a source of contention:

Every CPU funnels through one lock. That single point is what the multi-queue design was built to remove.

Each CPU had to coordinate through the same lock and frequently modify the same shared cache lines. The software queue could become a bottleneck before the device reached its potential.

The kernel needed a design that preserved request management while allowing submission to scale across CPU cores and hardware queues.

Linux's modern block layer uses blk-mq, the multi-queue block I/O architecture.

It has two important queue levels:

The diagram shows one possible mapping. A hardware dispatch queue is not guaranteed for every CPU. The number and mapping depend on the driver, device, CPU topology, and queue configuration.

Software staging queues

Software staging queues provide CPU-local or node-local entry points. They reduce the need for every submitting CPU to modify one global queue under one lock.

Requests may enter a staging queue when the block layer is attempting to merge work or when an I/O scheduler is active. The scheduler can organize requests before they become eligible for hardware dispatch.

When no scheduling or merging work is needed and the device has capacity, blk-mq can attempt a more direct path toward the driver.

Hardware dispatch queues

A hardware dispatch queue represents a path through which the driver can send requests to the device.

An NVMe controller can expose multiple submission queues. The NVMe driver can map blk-mq hardware contexts onto those queues, allowing requests from different CPUs to proceed with less shared contention.

A device with less hardware parallelism may expose fewer queues, causing several software queues to map to the same hardware dispatch queue. Multi-queue means the architecture supports multiple queueing paths; it does not promise a one-to-one CPU-to-device-queue mapping.

If the device temporarily cannot accept another command, a request must remain pending until resources become available.

Tags identify in-flight requests

Once several requests can execute concurrently, completion order is not enough to identify which one finished.

blk-mq assigns a numeric tag to an in-flight request. The driver and completion path use that tag to find the corresponding request directly.

If tag 19 completes first, the kernel can complete that request without scanning the entire queue or waiting for tags 7 and 4.

Tags also represent finite in-flight capacity. When no suitable tag is available, more work must wait before it can be dispatched.

An I/O scheduler is an optional policy layer within blk-mq. The schedulers available for a device depend on the running kernel's configuration and what that device supports.

Current Linux systems can expose the following choices:

These are choices, not a ranking. A scheduler that helps one device and workload can add overhead or reduce throughput for another.

none

none avoids applying an I/O scheduling policy that reorders requests.

It does not bypass the block layer. Requests still use blk-mq, obey device limits, receive tags, and pass through the driver. Merging can still occur elsewhere in the submission path, and the device itself may reorder commands.

none is often a reasonable baseline for fast NVMe devices, virtual disks, or storage stacks where another layer already performs substantial scheduling.

mq-deadline

mq-deadline organizes requests partly by logical sector so nearby work can be dispatched efficiently. It also assigns expiration times so a request is not postponed indefinitely for the sake of better ordering.

Reads normally receive preference because applications often wait directly for read results. Writes cannot be ignored forever; the scheduler limits how long preference can continue before write work is dispatched.

The result is a compromise between throughput, latency, and starvation avoidance. The word “deadline” means the scheduler attempts to begin service within its policy window, not that it provides a hard real-time completion guarantee.

Kyber

Kyber is designed around latency targets. It limits how many requests of particular classes are admitted so the device queue does not become so deep that latency grows uncontrollably.

Its exposed targets cover reads and synchronous writes. Kyber adjusts throttling based on observed completion behavior.

This approach is most relevant when the device has substantial parallelism but excessive queueing would damage latency. Kyber may not be built or offered for every device.

BFQ

Budget Fair Queueing, or BFQ, associates work from processes or groups with budgets measured in sectors. It aims to distribute device bandwidth proportionally while providing low latency for interactive or time-sensitive work.

BFQ can improve responsiveness when background I/O competes with an interactive application. Its stronger accounting and service policy also add per-request overhead. Maximum throughput and strongest fairness are not always achieved by the same configuration.

BFQ is therefore most useful when workload isolation, proportional sharing, or interactivity is worth additional scheduling work.

Linux exposes block-layer properties through sysfs. Start by listing whole block devices:

For a device named nvme0n1, inspect its scheduler:

Example output:

The scheduler in brackets is active. The other names are available choices on that system. Another machine may show fewer choices because of its kernel configuration or device.

Several queue properties can be inspected without changing them:

These values describe:

nr_requests is not simply the controller's hardware queue depth. Its scope depends on blk-mq tag sets, hardware-queue count, the active scheduler, and whether request pools are shared.

The exact files can vary by kernel and device. A partition may refer back to queue properties on its parent disk, and a virtual device may report capabilities chosen by a lower storage layer rather than physical hardware.

Reading these properties is safer than guessing from a device name. nvme0n1 reveals a protocol, while the queue files reveal the limits and policy Linux is actually using.

Suppose three kernel callers submit:

A possible block-layer path is:

Another device or scheduler could make different decisions. A small maximum request size might prevent A and B from merging. none would not apply scheduler reordering. A controller with one hardware queue exposes less dispatch parallelism than a multi-queue NVMe controller.

The invariant is not a particular request count or order. The invariant is that the block layer must respect operation semantics and device constraints while managing requests efficiently.

There is no universally fastest I/O scheduler.

For a rotating HDD, reordering nearby requests can save expensive seeks. A policy that prevents starvation is also valuable when several processes generate random I/O.

For a fast NVMe SSD, complex host-side ordering may cost more CPU time than it saves in device work. none can be effective when the controller handles parallel requests well, while mq-deadline or Kyber may help when tail latency under load matters.

For an interactive workstation or a shared server, BFQ's bandwidth sharing can keep one background job from making other work unresponsive.

For a virtual or cloud block device, the guest kernel may see only a virtual queue. The host or storage service can perform another layer of scheduling that the guest cannot observe. A sophisticated guest policy may duplicate work or make decisions using incomplete information.

The practical method is:

Scheduler names suggest design goals. They do not predict the outcome without the device and workload.

The block layer accepts block I/O from higher kernel layers and converts it into requests suitable for device drivers. A bio describes an operation from above, while a request packages one or more bios for scheduling and dispatch. Requests may be split to satisfy device limits or merged to reduce command overhead.

Linux blk-mq scales submission with software staging queues mapped to hardware dispatch queues. Tags identify concurrent in-flight requests, and completion order does not have to match submission order. NVMe can expose several hardware queues, while simpler or virtual devices may provide fewer.

I/O schedulers apply optional dispatch policies. none minimizes policy overhead, mq-deadline balances locality with expiration times, Kyber manages admission around latency targets, and BFQ emphasizes proportional bandwidth and responsiveness. The appropriate choice depends on the device, queue topology, workload, and whether the goal is throughput, latency, or fairness.