ExecutorService
Executor Service
Creating a new thread for every task sounds simple. But it's a terrible idea at scale.
Threads are expensive. Each one consumes memory for its stack (typically 1MB), and the OS spends CPU cycles managing them. Create thousands of threads, and your application grinds to a halt.
Executor Service solves this by decoupling task submission from task execution. You submit tasks; a pool of reusable threads executes them. No more manual thread management.
In this article, we'll explore:
- Why manual thread creation doesn't scale
- How executor services work
- Types of thread pools
- Submitting tasks and getting results
- Proper shutdown and lifecycle management
- Thread pool sizing strategies
- Common patterns and pitfalls
If you're enjoying this newsletter and want to get even more value, consider becoming a [paid subscriber](https://blog.algomaster.io/subscribe).
As a paid subscriber, you'll unlock all premium articles and gain full access to all [premium courses](https://algomaster.io/newsletter/paid/resources) on [algomaster.io](https://algomaster.io).
1. The Problem with Manual Thread Creation
Consider a web server handling requests:
This approach has serious problems:
With 10,000 concurrent requests, you'd create 10,000 threads consuming ~10GB just for thread stacks. The system would thrash from context switching and likely crash.
2. What is an Executor Service?
An Executor Service manages a pool of worker threads that execute submitted tasks. Instead of creating threads directly, you submit tasks to the executor, which handles all thread management.
Key Benefits
Basic Usage
3. How Executor Service Works
An executor service consists of three main components:
The Execution Flow
- Client submits a task to the executor
- Task is placed in the queue (if no idle worker)
- An idle worker picks up the task from the queue
- Worker executes the task
- Worker returns to idle state or picks up next task
4. Types of Thread Pools
Different workloads need different thread pool configurations. Most languages provide factory methods for common patterns.
Fixed Thread Pool
A pool with a fixed number of threads. If all threads are busy, tasks wait in an unbounded queue.
Use when: You know the optimal thread count and want predictable resource usage.
Cached Thread Pool
Creates threads as needed and reuses idle threads. Threads expire after 60 seconds of idleness.
Use when: Tasks are short-lived and you have variable load. Caution: Can create unlimited threads under high load.
Single Thread Executor
A pool with exactly one thread. Tasks execute sequentially in submission order.
Use when: Tasks must execute sequentially, or you need a dedicated background thread.
Scheduled Thread Pool
Executes tasks after a delay or periodically.
Use when: You need delayed or periodic task execution (timeouts, polling, heartbeats).
Comparison
5. Submitting Tasks
There are two ways to submit tasks: execute() for fire-and-forget, and submit() when you need a result or exception handling.
Runnable vs Callable
execute() vs submit()
6. Future: Getting Results
When you submit a task, you get a Future object that represents the pending result.
Future Methods
Basic Usage
Handling Exceptions
Exceptions thrown by tasks are wrapped in ExecutionException:
Waiting for Multiple Tasks
Or use invokeAll():
7. Shutdown and Lifecycle
Proper shutdown is critical. An executor that isn't shut down keeps its threads alive, preventing the JVM from exiting.
shutdown() vs shutdownNow()
Proper Shutdown Pattern
8. ThreadPoolExecutor: Custom Thread Pools
The factory methods use ThreadPoolExecutor internally. For fine-grained control, create it directly.
How Core and Max Pool Size Work
Queue Types
Example: Bounded Thread Pool
9. Rejection Policies
When the queue is full and max threads are busy, the executor must reject new tasks. The rejection policy determines how.
Policy Comparison
CallerRunsPolicy: Built-in Back-Pressure
CallerRunsPolicy is particularly useful. When the pool is saturated, it makes the submitting thread execute the task itself, slowing down the producer.
10. Thread Pool Sizing
Choosing the right pool size depends on your workload.
CPU-Bound Tasks
Tasks that compute without blocking (mathematical calculations, data processing).
Optimal threads = Number of CPU cores
More threads would cause context switching overhead without benefit.
I/O-Bound Tasks
Tasks that wait for I/O (network calls, disk reads, database queries).
Optimal threads = CPU cores × (1 + Wait time / Compute time)
If tasks spend 90% of time waiting, you can use 10x the cores.
Sizing Guidelines
11. Common Patterns
Pattern 1: Web Server Request Handler
Pattern 2: Parallel Processing with Results
Pattern 3: Timeout for External Calls
Pattern 4: Periodic Health Check
Pattern 5: Graceful Shutdown with Hook
12. Common Pitfalls
Pitfall 1: Never Shutting Down
Pitfall 2: Unbounded Queue with Fixed Pool
Pitfall 3: Ignoring Exceptions in Tasks
Pitfall 4: Blocking in Cached Thread Pool
Pitfall 5: Using Default Thread Factory in Production
13. Summary
Executor Service provides a high-level abstraction for managing thread pools and executing tasks asynchronously.
Thread Pool Selection
Sizing Guidelines
Best Practices
- Always shut down executors to prevent resource leaks
- Use bounded queues to prevent memory issues
- Choose appropriate rejection policy for your use case
- Size pools based on workload (CPU vs I/O bound)
- Use custom thread factories with meaningful names
- Handle exceptions from tasks (use Future.get())
- Prefer submit() over execute() for better error handling
- Use CallerRunsPolicy for natural back-pressure
- Add shutdown hooks for graceful termination
Executor services are fundamental to building scalable concurrent applications. Understanding their internals helps you configure them correctly and avoid common pitfalls.
Thank you for reading!
If you found it valuable, hit a like and consider subscribing for more such content every week.
If you have any questions or suggestions, leave a comment.
This post is public so feel free to share it.
P.S. If you're enjoying this newsletter and want to get even more value, consider becoming a [paid subscriber](https://blog.algomaster.io/subscribe).
As a paid subscriber, you'll unlock all premium articles and gain full access to all [premium courses](https://algomaster.io/newsletter/paid/resources) on [algomaster.io](https://algomaster.io).
There are [group discounts](https://blog.algomaster.io/subscribe?group=true), [gift options](https://blog.algomaster.io/subscribe?gift=true), and [referral bonuses](https://blog.algomaster.io/leaderboard) available.
Checkout my [Youtube channel](https://www.youtube.com/@ashishps_1/videos) for more in-depth content.
Follow me on [LinkedIn](https://www.linkedin.com/in/ashishps1/) and [X](https://twitter.com/ashishps_1) to stay updated.
Checkout my [GitHub repositories](https://github.com/ashishps1) for free interview preparation resources.
I hope you have a lovely day!
See you soon, Ashish
References
- Java Concurrency in Practice (Brian Goetz) - Chapter 6: Task Execution, Chapter 8: Applying Thread Pools
- Java ExecutorService Documentation
- Java ThreadPoolExecutor Documentation
- Effective Java (Joshua Bloch) - Item 80: Prefer executors, tasks, and streams to threads
Images Needed
[IMAGE 1: Thread-per-Task Problems] <!-- Shows memory explosion, context switching issues -->
[IMAGE 2: Executor Service Architecture] <!-- Shows task queue, thread pool, worker threads -->
[IMAGE 3: Thread Pool Types Comparison] <!-- Visual comparison of fixed, cached, single, scheduled -->
[IMAGE 4: Task Submission Flow] <!-- Shows how tasks flow from submit to execution -->
[IMAGE 5: Shutdown Lifecycle] <!-- State diagram: Running -> ShuttingDown -> Terminated -->
[IMAGE 6: Thread Pool Sizing Decision] <!-- Decision tree for CPU-bound vs I/O-bound sizing -->