The previous lesson covered thread pools and how reusing a fixed set of worker threads beats spawning a new Thread for every task. This lesson covers the API used to talk to a pool: the Executor interface, its richer cousin ExecutorService, the Executors factory class, and the lifecycle methods (shutdown, shutdownNow, awaitTermination) that decide when the program is allowed to exit. We'll build an order-confirmation pipeline for an online store, submit tasks to a pool, shut it down cleanly during a deployment, and look at what happens when the shutdown step is skipped.

The running example uses Runnable tasks so the API stays in focus.

A program that calls new Thread(task).start() for every job is doing too much work and getting too little control. It creates a thread per task (expensive), it has no way to limit how many threads run at once (dangerous), it has no way to wait for a batch of work to finish (awkward), and it has no clean way to shut down. The framework replaces all of that with two ideas: submit a task, and let something else decide how to run it.

Three orders, three brand-new threads. None of them gets reused. There's no central place to ask "are all orders done yet?" The order of the output is also non-deterministic, so on another run the lines could come out in a different order or the thread names could shift. Contrast that with the executor version:

Same three orders, only two threads. The first two tasks each grab a worker. The third task waits for one of the workers to come free, then runs on it. The structural point for now: a pool plus a submit-and-shutdown protocol is the basic shape of every executor program.

The diagram shows the model. Application code drops tasks into a queue. A small fixed set of workers picks them off and runs them. The threads are never wired up, named, or joined manually. The framework owns that wiring; the application owns the tasks and the lifecycle.

java.util.concurrent.Executor is the lowest layer of the API. It has exactly one method:

That's the whole contract. Hand it a Runnable, and the executor runs the runnable at some point, on some thread, in some manner of its choosing. The interface doesn't promise the task runs immediately, doesn't promise it runs on a different thread, doesn't promise concurrency at all. A legal Executor is one that runs every task on the calling thread:

The lambda assigned to inline satisfies the Executor interface (one abstract method, takes a Runnable). Its implementation calls command.run() synchronously on the caller's thread. The runnable prints first, and only then does control return to main to print the second line. That's a legitimate executor, though not a useful one for real work.

In practice, the bare Executor interface is rarely used directly. The pool implementations all expose ExecutorService, which extends Executor with the methods used in real code. The reason Executor exists as a separate interface is that some libraries accept the small contract on purpose, so they can be handed either a pool, an inline runner, or something exotic without depending on the wider API.

ExecutorService extends Executor with lifecycle management, task submission methods that return a Future, and bulk-submission helpers. The methods fall into three groups:

The submit and bulk-submit methods return Future objects. For this lesson the Runnable overload of submit is used, which returns a Future<?> whose only useful operation is "wait until the task finishes" or "check if it threw an exception." The lifecycle methods are the load-bearing pieces of the rest of this chapter.

Four orders, three workers. The first three each pick up an order and start the (simulated) email send. When pool-1-thread-1 finishes its 200 ms sleep, it picks up the fourth order. The exact thread names won't always match the example, because the scheduler decides which worker grabs the queued task, but the pattern is stable: workers churn through the queue until it's empty.

submit always returns a Future, even for Runnable tasks where the value is always null. Allocating that future has a small cost compared to bare execute. When completion, exceptions, or cancellation don't matter, execute is slightly cheaper. In real code, the safety of being able to inspect failures usually wins.

The submit versus execute choice is worth pausing on. With execute, if the runnable throws an exception, it's reported to the pool's UncaughtExceptionHandler (or printed to stderr by default), and the rest of the program never sees it. With submit, the exception is captured in the Future, and calling future.get() rethrows it as the cause of an ExecutionException. For order processing, that visibility matters: knowing which order failed is more useful than knowing that something failed somewhere.

Constructing a pool with new ThreadPoolExecutor(...) directly is rare. The Executors class has static factory methods that build the common pool shapes. The ones to know:

For most order-processing work, newFixedThreadPool is a sensible starting point.

The fixed pool spreads three tasks across two workers. The single-thread executor runs both of its tasks on the same worker (pool-2-thread-1) in submission order, because that's its whole job: serial execution with a queue in front. A single-thread executor fits a sequence of tasks that must not overlap, like writing audit lines to a single log file.

Both newFixedThreadPool and newSingleThreadExecutor use an unbounded queue. If tasks arrive faster than the pool can drain them, the queue grows without limit and the JVM eventually runs out of heap. In production, prefer a bounded queue (via the ThreadPoolExecutor constructor) so the pool refuses excess work explicitly rather than buffering it forever.

The cached pool is a different beast. It creates threads on demand and lets them die after 60 seconds of idle time. It works well for short, bursty, mostly-IO tasks. It works poorly for steady high-throughput workloads, because there's no upper bound on the number of threads it can spawn. A misconfigured cached pool can turn one second of traffic into a thousand live threads. For order processing under load, the fixed pool's predictable thread count is almost always the better default.

A more realistic shape. An online store places an order and needs three things to happen: an email to the customer, an SMS to the customer's phone, and an entry written to an audit log. Each is independent. Each takes a moment because it's a network call. A pool with three workers can do all three in parallel, then the main thread can move on.

Nine tasks, three workers, and the whole batch finishes in about half a second. The exact order varies between runs because the scheduler decides which task starts on which worker. What is stable is that each order's audit log appears before its SMS, which appears before its email, because the audit job sleeps the least and finishes first when all three run together. If the calls were sequential on a single thread, the same nine tasks would take about nine times 117 ms (the average), which is over a second. Spreading them across three workers cuts the wall-clock time to one-third.

Each placed order fans out to three tasks. They sit briefly in the executor's queue, then three workers pick them up and run them in parallel. The customer-facing thread (the one that called placeOrder) doesn't wait for the tasks; it returns immediately and lets the workers finish on their own time. The shutdown step at the end of main is what guarantees the program waits for the workers before exiting.

When a batch of tasks should be submitted as a unit, the bulk-submit methods are cleaner than a loop of submit calls. They take a Collection<Callable<T>> and block until the batch reaches the requested state.

invokeAll submits every task and blocks until they all finish (or are cancelled). It returns a List<Future<T>> in the same order as the input collection, so each task's result can be read out by index.

invokeAll doesn't return until all three tasks have either finished or been cancelled. After it returns, every future in the list is done, so calling get on them in a loop never blocks for long. The output preserves input order even though the tasks themselves ran in parallel.

invokeAny is the inverse. It submits every task and returns the result of the first one that finishes successfully, cancelling the rest. It fits when several ways exist to answer the same question and only one answer is needed. A common shape is querying two regional pricing services and accepting whichever responds first.

us-west sleeps for 80 ms, the others for 150 and 200. invokeAny returns the moment us-west finishes, and the other two tasks are cancelled. If the fastest task throws an exception, invokeAny retries with the next finisher; only if every task fails does it throw ExecutionException. The rest of the chapter won't lean heavily on these methods, but the takeaway is clear: instead of a loop of submits and a loop of future.get() calls, invokeAll does both in one line, and invokeAny handles the "race for first answer" pattern.

An ExecutorService has a finite, one-way lifecycle. Once it leaves a state, it never comes back. Five states matter:

The pool moves through these states by calling shutdown() (RUNNING to SHUTDOWN), shutdownNow() (RUNNING or SHUTDOWN to STOP), and then waiting for it to reach TERMINATED with awaitTermination().

The diagram shows the entire lifecycle. The two human-driven transitions are shutdown() and shutdownNow(). The two automatic transitions are "queue drained" and "tasks done." awaitTermination waits for the pool to reach the green TERMINATED node. The rest of this section walks through each transition with code.

shutdown(): Polite Refusal of New Work

shutdown() is the gentle one. It moves the pool from RUNNING to SHUTDOWN. After the call:

shutdown() returns immediately. It doesn't wait for anything. To wait, call awaitTermination next.

shutdown() flips isShutdown to true right away, but the pool isn't terminated yet because four tasks are still queued or running. The late submit is rejected with an exception, exactly as advertised. Meanwhile, the four queued tasks keep running and printing. After awaitTermination returns, every task is done and isTerminated is now true.

The distinction between isShutdown and isTerminated is a frequent source of confusion. isShutdown is true as soon as shutdown() or shutdownNow() is called, even if work is still in progress. isTerminated is true only once the pool has actually finished everything and reached the TERMINATED state. Check isShutdown to ask "can submission still happen?" and isTerminated to ask "is the pool fully done?"

awaitTermination: Wait With a Timeout

awaitTermination(timeout, unit) blocks the calling thread until one of three things happens:

The timeout is mandatory. There's no "wait forever" version, and that's deliberate. A wait-forever call is a deadlock waiting to happen. A reasonable shutdown loop uses a short, sane timeout and escalates if the pool doesn't terminate.

The first awaitTermination returns false because the task is still sleeping at 300 ms. The second call returns true once the task wakes up and the pool drains. In a real shutdown loop, the false return is your cue to escalate.

shutdownNow(): The Forceful Stop

shutdownNow() is the hammer. It moves the pool to STOP. After the call:

The interrupt rule is more consequential than it appears. An interrupt is a request, not a kill. If task code is while (true) { doStuff(); } and never checks Thread.currentThread().isInterrupted() and never calls a blocking method like Thread.sleep that throws InterruptedException, then shutdownNow() will set the interrupt flag and the task will keep running forever. The framework can't kill a thread that refuses to listen.

The first task gets through one iteration, prints Step 0, and then the interrupt arrives while it's sleeping. Thread.sleep throws InterruptedException, the catch block runs, and the task exits cleanly. The two queued tasks never run; shutdownNow returns them in a list so the caller can decide what to do (retry them on a different pool, log them, drop them). The pool then transitions to TERMINATED.

The "return the not-run tasks" detail is useful in deployments. During a graceful restart, calling shutdownNow() near a deadline and re-queuing the returned tasks onto a different pool keeps work from being lost.

A reliable shutdown pattern uses both shutdown and shutdownNow. The shape is:

All six confirmations finish well inside the 5-second grace window, so awaitTermination returns true and shutdownNow is never called. That's the happy path. To see the unhappy path, drop the grace from 5 to 1. The first batch of three finishes (the tasks each sleep 300 ms), but the second batch doesn't have time, so shutdownNow kicks in and interrupts the running tasks. The interrupt message shows up in the output, the pool reaches TERMINATED, and the program exits.

The shutdownAndAwait helper is useful to copy into a util class. Every long-lived service that uses pools needs this dance somewhere. Writing the helper once saves debugging time later when something stops draining cleanly during a deploy.

Since Java 19, ExecutorService extends AutoCloseable. The default close() method does exactly the "graceful then forceful" dance: it calls shutdown(), waits a while, calls shutdownNow() if the wait expires, and propagates interrupts. That makes try-with-resources viable for executors in the same way it has always been for streams.

There's no explicit shutdown in this program. The closing brace of the try triggers close(), which runs the graceful shutdown internally and waits for all submitted tasks to finish. The "Pool closed" line is guaranteed to print after every task has run.

Caveats on the try-with-resources style:

For one-off scripts and short-lived programs, try-with-resources is the cleanest version of the pattern. For long-running services, the explicit shutdownAndAwait helper still wins.

A common bug in the executor API. A pool is built, tasks are submitted, the tasks finish, and the JVM never exits. The program hangs.

What's wrong with this code?

Output (then the program hangs):

The two confirmations print, the "All tasks submitted" message prints, and then the program sits there. Running it in a terminal requires Ctrl+C to kill it. The reason is that the pool's worker threads are non-daemon by default, and the JVM doesn't exit until every non-daemon thread has exited. The workers are idle, waiting for more tasks from the queue, but they're alive. As far as the JVM is concerned, "non-daemon thread still running" means "program still running."

Fix:

The shutdown() call signals the workers to exit once the queue drains. awaitTermination blocks the main thread until the workers have actually gone away. After that the JVM exits naturally because there are no non-daemon threads left.

A pool with no shutdown is the executor equivalent of a leaked file handle. In a short program it hangs on exit. In a long-running service, repeatedly creating pools without shutting them down leaks threads, and threads have real memory and OS costs. A leaked pool of size 32 means 32 OS threads held forever for nothing.

The mistake is easy to make because the pool runs the tasks just fine without shutdown(). The tasks finish, the program looks like it worked, and only the exit hang reveals the bug. Make a habit of writing the shutdown right when the pool is created. On Java 19+, try-with-resources handles it automatically. On an older JDK, the shutdownAndAwait helper is the standard pattern.

A pool can reject a submission for two reasons: the pool has already been shut down, or the pool has a bounded queue and the queue is full. Bounded queues come from new ThreadPoolExecutor(...) rather than the Executors factory methods we've used so far, but rejection still applies to the factory-built pools because of the shutdown case.

RejectedExecutionHandler is the interface that decides what happens when a submission is rejected. It has one method, rejectedExecution(Runnable r, ThreadPoolExecutor executor), called by the pool when it can't accept the task. The JDK ships four built-in policies as nested classes of ThreadPoolExecutor:

Custom policies are also possible. The signature provides both the task and the executor, so a custom policy can log, increment a metric, write the task to a dead-letter file, or do something else.

The default AbortPolicy threw RejectedExecutionException from the post-shutdown submit. In real code, that exception is usually a signal that the calling code is doing something wrong (submitting work after the pool has been retired), so the abort behavior is a reasonable default. Pools with bounded queues, where rejection can happen as a steady-state backpressure mechanism, often configure CallerRunsPolicy instead so the caller naturally slows down when the workers can't keep up.

The takeaway is that the framework has a defined rejection model, and RejectedExecutionException should be known so it can be handled where it might appear.

To pull the lifecycle pieces together, the kind of code that runs in an order-processing service during a rolling deployment. The service has been told to drain (stop accepting new work), and it's working its way through whatever's still in flight before exiting.

Eight orders, four workers, two rounds of 200 ms each. The drain signal arrives after the first 50 ms (the four workers are still in their first sleep), and shutdown followed by awaitTermination(3 seconds) is plenty of time to finish the second batch. The pool returns true from awaitTermination, the helper returns true, and the service prints "drained cleanly" before exiting.

Running the same program with the work-time bumped up (say Thread.sleep(1000) per order) and the grace tightened to one second shows the helper fall through to shutdownNow, interrupt the running tasks, and report a clean termination via the hard-stop wait. That's the contract: the pool always terminates, eventually, but whether it does so gracefully or forcefully depends on the timeouts chosen.