The previous chapter described a system call as a controlled request from an application to the kernel.

Now we will follow that request across the protection boundary.

Consider this C statement:

It looks like an ordinary function call. Underneath, the program prepares a kernel request, enters privileged execution, switches to a protected kernel stack, runs kernel code, and eventually returns to the instruction after read.

The complete path is:

The exact instructions and register names depend on the processor architecture. This chapter uses Linux on x86-64 as its concrete example while keeping the overall model applicable to other systems.

The application begins with a normal C function call:

The read function is usually provided by the C library. Its wrapper translates the C calling convention into the Linux system-call convention.

On x86-64 Linux, the wrapper arranges values like this:

The wrapper then executes the architecture's controlled kernel-entry instruction.

The application is still running in user mode while these registers are prepared. Merely placing a system-call number in rax does not give the process additional privilege.

On modern x86-64 Linux, the entry instruction is named syscall.

On AArch64, Linux normally uses svc, which means supervisor call. Older x86 systems commonly used a software-interrupt mechanism such as int 0x80.

Operating-systems textbooks often describe this event as a trap into the kernel. It is synchronous and intentional: the running program executes a specific instruction because it wants a kernel service.

It is not a hardware interrupt arriving from an external device.

When the CPU executes the entry instruction, hardware performs a tightly controlled transition. It changes to a privileged execution level and transfers control to an entry address configured by the kernel.

The application cannot select an arbitrary kernel function as the destination. The CPU goes to a predefined entry point, where the kernel takes control of the request.

On x86-64, the processor also preserves essential return information in designated registers. This includes the address of the user instruction to resume and selected processor flags.

At this moment, execution has entered kernel mode, but the requested file operation has not started yet. Architecture-specific entry code must first establish a safe environment for ordinary kernel code.

While the application runs, its stack pointer refers to a user stack in the process's user-space address range.

The kernel cannot safely use that stack for privileged work. The user stack may be nearly full, deliberately corrupted, unmapped, or changed by another user-space execution context.

Linux therefore associates a separate kernel stack with each schedulable thread.

The two stacks coexist. Entering the kernel does not copy the entire user stack into kernel memory.

Instead, the entry code records the user stack pointer and changes the CPU's stack pointer to the top of the current thread's kernel stack. Kernel functions then use this protected stack while handling the request.

The details vary by architecture and entry mechanism. In particular, the x86-64 syscall instruction does not automatically load a new stack pointer. Linux's carefully written entry code performs that switch before using the stack for normal kernel execution. Other entry mechanisms may receive more hardware assistance.

The kernel stack provides trusted space for:

Kernel stacks are intentionally limited in size. Kernel code cannot assume that it has the large, automatically growing stack that an application may appear to have.

The kernel must remember enough state to continue the application later.

The saved state includes values such as the user instruction pointer, user stack pointer, processor flags, and general-purpose registers.

On Linux, the entry code organizes much of this information in a register frame commonly represented by a structure named pt_regs.

Conceptually, the kernel stack now holds the kernel's own call frames, and beneath them the saved user state:

Saving state serves two purposes.

First, kernel code is free to use processor registers without destroying the application's values.

Second, the saved frame gives the return path the information needed to resume user execution safely.

This is more than the return-address handling performed by an ordinary function call. The kernel is preserving state across a privilege transition between separately protected environments.

The earliest kernel-entry code is architecture-specific and often written partly in assembly language.

Its job is to turn the hardware-specific entry state into a form that common kernel code can use. It establishes kernel data access, saves registers in the expected layout, and applies entry-time security rules required by that architecture.

Only after this setup can the kernel treat the request like an ordinary kernel operation.

This boundary code must be exceptionally careful. It executes before a full, conventional kernel call frame exists, and every value arriving from user space is untrusted.

The progression is architecture-specific entry, then a safe kernel stack with saved state, and finally the common system-call handling code that the rest of the kernel shares.

Modern processors and kernels include additional security work on entry and exit. Its performance impact will be discussed in the next chapter; the important point here is that entering privileged code requires more setup than calling a function in the same process.

The kernel reads the system-call number supplied by the application.

It checks that the number identifies a supported operation for the current ABI, then dispatches to the corresponding kernel handler.

Conceptually:

The kernel uses the system-call number to select an entry from its dispatch table, and each entry names the handler that implements that operation:

The real Linux implementation includes generated tables, architecture-specific wrappers, and common kernel functions rather than one simple source-code array matching this diagram. The logical operation is the same: select a known handler without allowing user code to choose an arbitrary kernel address.

An unsupported number results in an error such as ENOSYS.

At this point, the kernel knows that the application requested read, but it still does not assume the request is valid.

The read request contains three arguments:

Each requires a different kind of validation.

Validating the file descriptor

The kernel checks whether fd refers to an open object in the calling process and whether that object supports reading.

An invalid descriptor can produce EBADF.

Validating the byte count

The kernel checks the requested size against limits relevant to the operation. It must reject impossible sizes and avoid arithmetic overflow when calculating memory ranges.

An invalid size or combination of flags can produce an error such as EINVAL.

Validating the user buffer

The buffer argument is an address in the calling process's virtual address space. It is not automatically a safe kernel pointer.

For read, the kernel eventually needs to place data into that user-space memory. It uses special user-access mechanisms to check and transfer data across the boundary.

The data moves in one direction, and every byte of that move is checked, because the destination address came from the process rather than from the kernel.

Linux provides helpers commonly described by names such as copy_to_user and copy_from_user. These helpers account for the fact that a user address may be invalid or may fault while being accessed.

If the destination is not writable, the request can fail with EFAULT instead of allowing an unsafe access to crash or corrupt the kernel.

Validating permission

The kernel also applies relevant access-control rules. These may depend on the process's credentials, the opened object's permissions, security policies, or the operation being requested.

Validation is syscall-specific. A networking call checks different objects and constraints from a memory-management call.

The general rule is always the same:

Every system-call argument originates from an untrusted process and must be interpreted safely before it influences privileged state.

Validation bugs are particularly dangerous because they can turn an ordinary system call into a way to read kernel data, overwrite protected memory, or gain additional privilege.

After validation, the selected kernel subsystem performs the requested operation.

For read, the kernel follows the file descriptor to the relevant open object and asks the appropriate file, device, pipe, or network implementation for data.

The path runs from the read handler, to the open object identified by fd, to the relevant kernel subsystem, and finally to the file, device, pipe, or socket data itself.

The implementation depends on what the descriptor represents. This is one reason the same read interface can work with regular files, terminal input, pipes, and some devices.

If data is immediately available, the operation may complete while the calling thread remains on the CPU.

If the operation must wait, the kernel can suspend the calling thread and allow another thread or process to run. When the needed event occurs, the original thread can resume inside the kernel and continue the same system call.

The process's user-space code does not run during this wait. The kernel stack and saved state preserve where its kernel-side execution should continue.

Scheduling and blocking I/O are covered in later modules. Here, the important point is that a system call begins synchronously but does not necessarily finish immediately.

When the handler finishes, it produces either a successful result or an error.

For read, a successful result is the number of bytes transferred. A result of zero can indicate end-of-file. A failure is represented internally by a negative error code.

Linux places the raw result in the architecture's return-value register. On x86-64, that register is rax.

The kernel also prepares to leave privileged execution. Before returning, it may need to account for pending work associated with the current task, such as a scheduling decision or signal delivery.

Those mechanisms can change exactly when and how user execution resumes, but they do not change the basic system-call contract: the application eventually receives a result or is otherwise notified by the operating system.

The return path reverses the essential work performed during entry.

Conceptually, the kernel:

On x86-64, Linux can use instructions such as sysretq or iretq, depending on the circumstances. AArch64 commonly uses eret.

The CPU does not return directly to an address supplied without checks by the application. The kernel verifies that the saved state is appropriate for user mode and uses an architecture-defined return mechanism.

The thread's kernel stack is left ready for its next kernel entry. User-space execution continues on the original user stack.

The processor resumes inside the C library wrapper immediately after its system-call entry instruction.

The wrapper examines the raw result.

If the result represents success, the wrapper returns the value to the application.

If the result represents a kernel error, the wrapper typically converts it to the function's documented failure value and sets errno.

The kernel returns one value. The wrapper turns it into the two-outcome interface that C programs expect, which is why application code checks for -1 rather than for a negative error code.

The original C statement can now finish:

If count is nonnegative, it reports how many bytes were read. If it is -1, the application checks errno to understand the failure.

From the source code's perspective, read behaved like a function. Internally, its execution crossed into a different privilege level, used a different stack, and ran code belonging to the kernel.

The complete sequence can now be assembled:

In user mode:

The protection boundary is crossed. In kernel mode:

The boundary is crossed again. Back in user mode:

Not every syscall performs every service-specific action shown here. For example, getpid does not transfer a data buffer, and close does not read from a device.

The entry, dispatch, validation, execution, and return pattern remains broadly applicable.

A system call always involves a transition from user-mode execution to kernel-mode execution.

This is a mode switch, also called a privilege-level transition.

It does not automatically mean that the CPU switches to a different process.

In the common fast path, the same thread enters the kernel, handles the request, and returns. The kernel is not necessarily a separate process receiving the request. It is privileged code executing on behalf of the calling thread.

A context switch occurs when the CPU stops running one thread or process and starts running another. The two are easy to confuse because one can lead to the other:

The same thread runs throughout the first path, changing only its privilege level. The second path ends with a different thread on the CPU. A blocking system call may lead to a context switch, and the kernel may also reschedule for other reasons. The system call itself requires only the mode switch.

This distinction explains why a quick call such as retrieving a small piece of kernel state can enter and leave the kernel without another process ever running.

The two operations can now be compared more precisely:

A system call performs substantially more boundary work than an ordinary function call. The next chapter examines that overhead and the techniques operating systems use to avoid some kernel entries.

A system call starts as a C library call that places an operation number and arguments in architecture-defined registers. A special instruction then transfers execution from user mode to a predefined kernel entry point.

The kernel switches to a protected kernel stack, saves the user state, dispatches the request, validates every untrusted argument, and performs the requested service. It then prepares a result, restores the user state, and returns through a controlled transition to user mode. The library wrapper converts raw errors into the application's documented return convention and errno.

A system call always causes a mode switch, but it causes a context switch only if another task is selected to run.

The essential mental model is:

The same thread crosses into the kernel, executes privileged code on a protected stack, and returns to the exact user-space point where it left.