A running system usually has more work ready than its CPUs can handle at once. A build might be running alongside a database, a web server, a terminal, and several background services. Although all of them may be ready to work, a single logical CPU can execute only one instruction stream at a time.

The operating system must therefore keep answering one deceptively simple question:

Which runnable thread should use this CPU next?

The part of the kernel that makes this decision is the CPU scheduler. It distributes CPU time among runnable threads, creating the useful illusion that many programs are progressing at once, even when there are more runnable threads than the hardware can execute simultaneously.

A computer can have thousands of threads, but it has only a limited number of logical CPUs. If a machine has four logical CPUs, at most four threads can execute instructions at exactly the same instant.

The rest of the threads may be:

Scheduling matters when the number of runnable threads exceeds the CPU capacity available to them.

Consider three runnable threads on one logical CPU:

Threads A, B, and C take turns on the one CPU, in the order A, B, C, A, B.

A, B, and C do not execute simultaneously. Their execution intervals are interleaved. If the intervals are short enough, a person or application interacting with them observes concurrent progress.

On a machine with multiple logical CPUs, some work can truly run in parallel:

Each CPU still runs one thread at a time, but two threads now genuinely execute at the same instant.

Scheduling is still required because the runnable population can exceed the number of CPUs, and because work continually blocks, wakes up, starts, and finishes.

Concurrency means that multiple computations make progress during overlapping periods. Their instructions may be interleaved on one CPU.

Parallelism means that multiple computations execute at the same instant on different CPUs.

A scheduler enables concurrency even on a single-CPU system. Multiple CPUs add the possibility of parallel execution, but they do not eliminate the need to choose among competing runnable threads.

This distinction explains why a process listing can show hundreds of active tasks on an eight-CPU server. The tasks exist concurrently, but no more than eight of their threads can execute at one instant.

The scheduler does not choose from every thread in the system. A thread waiting for disk data, a network packet, a timer, or a lock cannot make progress merely because it receives CPU time.

The scheduling population consists of threads that are eligible to execute now. These are called runnable threads. Conceptually, the kernel keeps them in a ready set:

Threads circulate rather than travel in a straight line. A thread that blocks rejoins the runnable set once its event completes, and the scheduler chooses again.

Real kernels use carefully designed scheduling data structures rather than one literal queue containing every runnable thread. The mental model remains useful:

When no eligible thread can run on a CPU, the kernel runs an idle task or places the processor into an appropriate low-power state until work arrives.

The operating system normally schedules threads, because threads are the independently executable instruction streams. Saying that the scheduler “runs a process” is convenient shorthand when that process has one thread.

Two kernel responsibilities are closely related:

The transfer may require saving the outgoing thread's execution context, restoring the incoming thread's context, and changing other CPU state. That mechanism is a context switch.

A scheduling decision does not always cause a switch to a different thread. The scheduler may determine that the current thread should continue. Conversely, when the current thread blocks or exits, some other runnable thread must be selected if useful work is available.

This separation is important:

Scheduling is the policy decision; dispatching is the mechanism that applies it.

The current owner of a CPU does not remain the obvious choice forever. The kernel may reconsider what should run when:

These events do not all guarantee that a different thread will run.

Suppose thread A is running when network data wakes thread B. B becomes runnable, but the scheduler may let A continue for now. Alternatively, it may decide that B should run immediately and preempt A.

If A blocks and B is the only other runnable thread, the decision is straightforward: B can use the CPU. If A, B, C, and D are all runnable, the scheduling policy must choose among them.

A simple system could allow a thread to keep the CPU until it blocks, exits, or voluntarily gives it up. This is a non-preemptive or cooperative approach.

It works only if running code cooperates often enough.

Consider a thread performing an endless calculation:

The loop does not wait for input, sleep, exit, or voluntarily yield. On a purely cooperative single-CPU system, it can prevent every other ordinary task from running.

Even a finite computation can be disruptive. A video encoder that computes for several seconds at a time may be making valid progress, but a terminal or request handler should not have to wait for the encoder to finish its entire computation.

General-purpose operating systems therefore use preemptive scheduling. The kernel can take a CPU away from a still-runnable thread and allow another thread to use it.

The total work is the same in both rows. Only the order and the size of the pieces change, and that is enough to decide how long B waits for its first turn.

Preemption does not mean the outgoing thread did anything wrong. It remains runnable and can continue later from its saved execution state.

Preemption requires the operating system to regain control even when the running program never asks for a kernel service.

A programmable hardware timer provides that control. The kernel arranges for a future timer interrupt, returns to the selected thread, and knows that hardware will eventually transfer control back to an interrupt handler.

Conceptually:

The decision point exists only because the timer interrupt brought the kernel back. Without it, A would keep the CPU until it chose to give it up.

The timer interrupt creates a scheduling opportunity. It does not imply that the kernel must switch threads on every timer event.

Modern kernels also receive scheduling opportunities from many other events, including blocking operations and device interrupts that wake waiting work. The timer's essential contribution is that a CPU-bound user program cannot indefinitely prevent the kernel from reconsidering who owns the CPU.

Most applications do not need a CPU continuously. Their execution alternates between periods of computation and periods of waiting.

A period during which a thread executes instructions is called a CPU burst. A thread may then begin an operation that cannot complete immediately and enter an I/O wait or another kind of blocked wait.

A request-handling thread alternates between the two:

This behavior creates two broad workload patterns.

A CPU-bound thread spends most of its active lifetime computing. It tends to have long CPU bursts and remains runnable for extended periods. Examples include compression, encoding, and large in-memory calculations.

An I/O-bound thread frequently waits for external events and tends to use the CPU in shorter bursts. Examples include an idle shell and a server worker waiting for requests.

These labels describe observed behavior, not permanent identities. A database thread may be CPU-bound while evaluating a complex query and I/O-bound while waiting for storage. The same program can change behavior over time.

The scheduler sees the consequence of this cycle:

Scheduling would be much simpler if every job arrived together, ran for a known amount of time, and never blocked. Real workloads are dynamic, and the kernel generally does not know a thread's future CPU needs.

If every runnable thread eventually received the CPU and context switches were free, almost any order might appear acceptable. Neither assumption holds.

The choice affects several properties that users and services notice.

Responsiveness

An interactive task or request handler may need only a short CPU burst before it can produce visible progress. Making it wait behind a long computation can make the whole system feel unresponsive even though the CPU is busy.

Continued progress

A system should prevent one CPU-hungry task from indefinitely excluding other ordinary runnable work. Sharing does not necessarily mean that every thread receives identical treatment, but runnable work needs a defensible opportunity to progress.

Useful CPU work

When eligible work exists, leaving a CPU needlessly idle wastes capacity. At the same time, switching too frequently spends more time in kernel overhead and can disrupt useful cache state.

Different service needs

Not all work has the same operational importance. Playing audio, processing a latency-sensitive request, running a background backup, and performing an emergency system operation place different demands on scheduling.

Predictability

Some workloads care less about finishing as early as possible than about receiving CPU service within a dependable time window. General-purpose systems must balance this need against throughput and ordinary interactive use.

These goals can conflict. Keeping a long-running computation on the CPU may reduce switching overhead, while interrupting it sooner may improve the responsiveness of short work. Giving one task more CPU time can necessarily give another less.

There is therefore no universally best scheduling policy. A policy is useful only relative to the workload and the behavior the system is trying to provide.

Preemptive multitasking is often described as dividing CPU time into small slices. That is a useful starting model, but it can create two false expectations.

First, a thread may stop before using its available CPU time because it blocks or exits.

Second, the kernel does not have to divide time into identical turns for every runnable thread. Scheduling policy may account for task importance, past CPU service, responsiveness needs, and other constraints.

Real slices are uneven because threads block, wake, and get preempted at moments that have nothing to do with a tidy rotation.

The uneven timeline alone does not prove that the scheduler is malfunctioning. Threads may wake at different times, block early, or be entitled to different treatment.

The scheduler's task is not merely to rotate through names. It must continually allocate a finite execution resource as runnable work changes.

With several logical CPUs, the kernel may dispatch several threads at once. Each logical CPU still executes only one thread at an instant, so scheduling decisions remain local to CPU ownership:

With A, B, C, D, and E all runnable, CPU 0 runs A, CPU 1 runs B, and CPU 2 runs C. D and E stay runnable and wait for CPU time.

The operating system must also avoid a situation in which one CPU is idle while another has eligible work waiting. Moving work between CPUs can improve utilization, although movement may have hardware-locality costs.

Those additional choices make multicore scheduling more complex, but they do not change the basic problem: choose an eligible thread for each available CPU.

Suppose a service has one CPU available and two runnable workers:

If A keeps the CPU until compression finishes, B's request waits even though it needs very little computation. The CPU can be 100% utilized while the service still has poor response latency.

With preemptive scheduling, the kernel can interrupt A, let B perform its short burst, and later resume A:

The CPU runs A, pauses it to give B a short slice for its response, and then continues A.

This does not create more CPU capacity. B's progress delays A slightly. Scheduling determines how contention is distributed; it cannot make total CPU demand disappear.

This distinction is useful during incidents. High CPU utilization says that the processors are busy. It does not say which runnable work is waiting, how long it waits, or whether the request path receives CPU at the right times.

Linux exposes several snapshots of scheduling pressure. Start with the number of logical CPUs available to the current environment:

Then ask ps to show individual threads, the CPU on which each was last observed, and its current state:

A typical fragment might look like:

R means running or runnable on Linux. A human-facing snapshot cannot reliably distinguish a thread executing at that exact instant from one waiting in runnable state for its turn.

vmstat provides a system-wide view:

Its output begins with columns similar to these:

The r column reports runnable work, including work currently executing. If r remains noticeably larger than the available CPU count across repeated samples, runnable tasks are competing for CPU time. A single high sample is only a clue because work can wake and finish between observations.

Linux also publishes load information:

Example:

The first three values summarize load over different time windows. Linux load includes both runnable tasks and tasks in certain uninterruptible waits, so it is not a pure CPU-queue measurement. In the fraction 2/1437, the first value is a snapshot of runnable scheduling entities and the second is the number of scheduling entities currently known to the kernel.

No single number proves that scheduling is the source of a latency problem. Runnable counts become meaningful when compared with available CPUs, CPU utilization, and application behavior over the same interval.

On a Linux test machine, two busy loops can demonstrate scheduling on one logical CPU. Do not run this experiment on a shared production host.

First, check which CPUs the current shell may use:

Choose one CPU from the printed list. The following commands use CPU 0; replace 0 if it is not allowed in your environment.

Start two CPU-bound workers restricted to that CPU:

Install a cleanup action immediately:

Sample their states and CPU usage:

Both workers remain runnable, but they cannot execute on CPU 0 simultaneously. Across several samples, the operating system interleaves them. Their measured CPU shares may not be exactly equal, especially over a short run, because sampling and other system activity affect the observation.

Stop the workers and remove the trap:

The important observation is not a particular percentage. It is that both computations make progress despite competing for the same execution resource. That behavior depends on repeated scheduling and preemption.

CPU scheduling exists because runnable demand can exceed the number of logical CPUs. The scheduler chooses which runnable thread should own each CPU, while the dispatcher applies that decision. Waiting threads are excluded until their events occur; runnable threads either execute or wait for a turn.

Preemption prevents CPU-bound or uncooperative code from monopolizing a CPU. Timer interrupts ensure that the kernel can regain control, reconsider its choice, and interleave independent computations. This creates concurrency on one CPU and coordinates access to CPUs on multicore systems.

Scheduling does not create capacity. It determines how finite CPU time is distributed, affecting progress, responsiveness, efficiency, and predictability whenever tasks contend for execution.