Handling Multiple Clients with Threads
A blocking TCP server can be correct and still be unusable for multiple clients.
Consider a server that accepts one connection, waits for that client to send a complete request, sends a response, closes the socket, and only then calls accept() again. If the first client takes ten seconds to send its request, every other client waits even when the machine has plenty of unused CPU and memory.
Threads provide a direct way to separate those waits:
- One main thread owns the listening socket and accepts connections.
- Each active connection is assigned to one worker thread.
- A worker can block on its client socket without blocking the accept loop or other workers.
This chapter builds bounded threaded TCP echo servers in Python and Java. Both implementations use the same length-prefixed wire protocol and handle one request per connection. They cap the number of active workers at 100, reject excess connections explicitly, apply per-client timeouts, and protect shared connection statistics.
The goal is not merely to create threads. It is to establish clear ownership, bounds, cleanup, and synchronization rules around them.