A worker pool is a long-lived, fixed-size set of goroutines that pull jobs off a shared channel, do the work, and push results onto another channel. It's the standard Go primitive for bounding concurrency on something expensive: a database, an external API, a CPU-heavy transform. Fan-out is the basic shape for a single batch; this lesson treats the pool as a reusable resource with its own lifecycle (start, drain, close), proper sizing rules, graceful shutdown, and error handling.

Fan-out and a worker pool look similar on a whiteboard. The difference is intent and lifecycle. Fan-out is "I have a batch of N items, spawn goroutines, wait for all to finish, move on." A worker pool is "I have a long-running service. Keep K workers alive, feed them through a channel, and emit results until I tell you to stop."

The same code can implement either, but treating a pool as a resource (with a Start, Submit, and Stop lifecycle) gives you the right boundaries for production code. A pool that processes orders all day shouldn't be respawned per HTTP request.

Every worker pool has five moving parts:

The closer is the piece that often gets missed. It exists because closing the results channel is the only way the consumer knows the pool is done, and the close has to happen after every worker has stopped sending. The WaitGroup is how the closer learns that's true.

The solid arrows are the data path. The dashed arrows are the lifecycle signal: each worker reports Done on the WaitGroup when it exits, and the closer turns "all workers are done" into "results channel is closed." The producer's job is small but specific: send every job, then close the jobs channel. That close is what lets the workers' for job := range jobs loops exit cleanly. Without it, workers block forever, the WaitGroup never drains, the results channel is never closed, and the consumer hangs.

An online store needs to validate incoming orders. Each validation pretends to call stock, address, and payment services, and rejects orders with non-positive totals or IDs flagged by fraud.

Errors don't crash the pool. The worker returns a ValidationResult with an Err field, so the consumer decides what to do. The consumer's totals only become correct after for r := range results drains, which happens only after the closer closes results, which happens only after every worker has exited its for o := range jobs loop. The whole chain hinges on close(jobs) from the producer. The worker function itself is identical to a fan-out worker; the "pool-ness" lives in main.

There's no single right answer for N. The value depends on whether the work is CPU-bound, IO-bound, or somewhere between.

Exceeding the downstream limit costs you nothing and breaks things. If a database accepts at most 20 connections, a 100-worker pool means 80 workers are constantly waiting, or worse, getting errors. For CPU work, going above runtime.NumCPU() adds scheduler overhead with no parallelism gain.

A concrete sizing example. A service that hits Postgres (50 connections), calls a tax API (30 req/s), and does light JSON work should size N to 30, set by the tax API. A pool of 50 would produce 429 errors from the tax service. A pool of 5 would leave most of the API's allowed bandwidth unused.

An unbuffered jobs channel makes every jobs <- o block until a worker is ready to receive. That's free backpressure: the producer's pace is capped by the workers' consumption rate. Memory stays flat regardless of how many jobs the producer could potentially emit.

A buffered jobs channel changes the shape slightly. The producer can fill the buffer ahead of the workers, which smooths out bursty sources. If the producer sometimes generates a burst of 20 jobs in a millisecond and then nothing for a second, a buffer of 20 lets the workers stay busy during the gap. To use a buffered jobs channel, you just change one line:

Buffering doesn't change the concurrency cap. With 4 workers, at most 4 jobs are ever being processed at once. The buffer adds queuing capacity, not parallelism. If the workers' aggregate throughput is lower than the producer's emission rate, the buffer fills up and the producer blocks anyway, just later than it would have with no buffer.

A large buffer doesn't help the workers. A buffer of 10,000 with 4 workers gives you 4 workers and a 10,000-slot queue. Pick the buffer to match the burstiness of the producer, not the throughput you want.

The basic pool above shuts down when the producer closes jobs. That works when the producer knows it's done. In a long-running service, the producer might be a goroutine that reads from a queue or an HTTP handler that never naturally "ends." You need another way to tell the pool to stop.

The classic Go pattern is a done channel: an extra channel the workers also watch, and that any goroutine can signal by closing. When done is closed, every worker sees the closure and exits, regardless of whether there's still work in jobs.

The done channel is a broadcast: closing it causes every receive on <-done to unblock immediately and return the zero value. Every goroutine watching that channel sees the signal at the same time.

The worker has two select blocks, not one. The outer select picks between "got a job" and "got a shutdown signal." The inner select picks between "published a result" and "got a shutdown signal mid-publish." Without the inner one, a worker could be stuck on results <- ... forever if the consumer has already exited. The producer also watches done so it stops generating new jobs as soon as shutdown starts, and it closes jobs on its way out so workers that already have a job in hand finish cleanly.

In a real service, shutdown might be triggered by an OS signal (SIGTERM), a context cancellation, or an explicit pool.Stop() call. The pattern is the same: one closer, broadcast via channel close. context.WithCancel is a higher-level version; the done channel is the lowest-common-denominator building block.

The consumer of a pool ranges over the results channel until it closes. That gives you a natural way to aggregate (sum, count, stream into a sink) without needing any extra coordination beyond the closer goroutine. The order validator example earlier accumulated accepted and rejected counts that way: the for r := range results loop runs until the closer closes the channel, and totals become correct exactly when that happens.

If you need to preserve input order, tag each result with a sequence ID and either sort at the end or write into a pre-allocated slice by index. The worker pool version of that trick uses the same approach as fan-out.

Two kinds of "stop" can happen, and they aren't the same thing. A per-job timeout says "if this one job hasn't finished in 200ms, abandon it and move on." Other jobs keep running, the pool stays alive, only the stuck job is dropped. A pool-wide cancellation says "stop everything." Every worker exits, in-flight jobs are abandoned, the pool stops accepting work.

A done channel handles the second. The first usually wants a per-job context or a select with time.After. Mixing the two leads to bugs: a single slow job shouldn't take down the whole pool, and a shutdown signal shouldn't wait for all in-flight jobs to finish.

The per-job timeout pattern races the work against a deadline:

Jobs 2 and 4 exceed the deadline, so the worker abandons them and reports a timeout error. The other jobs run normally, and the pool keeps processing. A timeout on one job doesn't affect any other job.

The downside of this pattern is that the goroutine doing the actual work doesn't get notified. When the deadline fires, the worker moves on, but the helper goroutine started inside runWithTimeout keeps running until it finishes. For genuinely cancelable work (HTTP requests, long DB queries), use context.WithTimeout, which signals the work itself to stop.

A common bug: closing the results channel too early.

What goes wrong: the producer closes both jobs and results immediately after sending the last job. The workers haven't finished yet, so when the next worker tries results <- n * 2, it panics with "send on closed channel." The trap is thinking "I've sent everything, so I'm done; close both channels." But "I've sent everything" doesn't mean "the workers have processed everything." The producer doesn't own the results channel; the workers do. Only something that knows all workers have exited can safely close it.

Fix: keep the producer responsible only for close(jobs), and add a separate closer goroutine that closes results after the WaitGroup drains.

A channel should be closed by exactly one goroutine, and that goroutine must be sure no one else is still sending. For the jobs channel that's the producer. For the results channel that's the closer (which waits on the WaitGroup to be sure).

The opposite mistake: the consumer exits before draining the results, so workers parked on a send stay parked forever.

In a short-lived program the process exits when main returns, so the leak doesn't matter in practice. In a long-lived service, every "stop reading early" path leaves goroutines parked on a send. They never get garbage collected because the runtime considers them live. Over hours and days, the count climbs and the service runs out of memory. The fix is to drain results fully or wire up a done channel so workers know to give up.