AlgoMaster Logo

Creating Threads

Medium Priority22 min readUpdated September 21, 2026
Listen to this chapter
Unlock Audio

Python gives you two ways to spawn a thread (a callable target or a Thread subclass) and a small set of methods for starting them, waiting on them, naming them, and deciding whether they keep the program alive. This lesson covers those mechanics: how to create threads cleanly, how to launch a batch of workers, how to get values back from them, and how to use thread-local storage when each thread needs its own private data.

Two Ways to Create a Thread

The threading.Thread class is the main entry point. You can use it directly by passing a function to target=, or subclass it and override run. Both produce a thread; they differ in how you organise the code around it.

The function form is the most common and the easiest to read:

target is the function the new thread runs. args is a tuple of positional arguments. kwargs, if you need it, is a dict of keyword arguments. The thread object itself does nothing until you call start(). After start(), the function runs on a fresh OS thread, and the main thread is free to do something else.

The subclass form bundles the thread with its own state. You write a class that inherits from Thread, override run, and call super().__init__() from your constructor:

super().__init__() is mandatory. Without it, the parent class's machinery is uninitialized and start() raises RuntimeError. Override run, not start. start() is what creates the OS thread and arranges for run() to execute inside it. Overriding start bypasses the threading mechanism entirely and the code runs in the main thread.

Which form to use? Use the function form for one-off tasks, simple workers, or when spawning many threads with the same target and different arguments. Use the subclass form when each worker needs its own configuration, when results should be exposed as attributes, or when the setup logic is naturally tied to the worker.

If the worker is a pure function call, pass it to target. If the worker carries enough state that tracking it in a separate dict or closure would otherwise be needed, a subclass is usually cleaner.

Starting Threads with start()

start() is the method that makes the thread real. Before start, a Thread object is just a configuration: a target, some arguments, a name. After start, the OS has a new thread of execution running your function.

A Thread instance can only be started once. Calling start a second time raises RuntimeError:

To run the same work twice, build a new Thread instance for the second run. The cost of creating a Thread object is low (it's a Python object), but the OS thread it spawns has its own stack and a small kernel-side bookkeeping cost. For high counts of short-lived work, prefer a thread pool.

A subtler point: calling run() directly does not start a new thread. It just calls the method on the current thread:

run() is a regular Python method. Only start() arranges for the OS to create a new thread and invoke run from inside it. Mixing the two up is a common early mistake, since worker.run() doesn't raise any error.

Each thread allocates a stack (typically 1-8 MB depending on the OS and Python version) and a small bookkeeping structure inside the kernel. Spawning hundreds of threads for short tasks is wasteful. The break-even point depends on how long each task runs, but a reasonable rule of thumb is to reuse threads through a pool once you're past a few dozen.

Waiting with join()

Once a thread is started, the main thread keeps running. If the main thread reaches the end of the script while a non-daemon worker is still running, Python will wait for the worker before exiting, but explicit waiting is usually better so the main thread can observe what the worker did.

join() blocks the calling thread until the target thread finishes:

The receipt line never prints before the charge completes, because join() blocks until the worker exits. This gives the main thread a way to wait for background work and then act on the result.

join() accepts an optional timeout in seconds. With a timeout, join returns after at most that many seconds, whether the thread finished or not. The return value is None either way, so checking is_alive() tells you which case applies:

This pattern is useful for showing progress, checking a cancellation flag, or refreshing a UI while a slow worker runs. The timeout doesn't speed anything up; it gives the main thread a chance to do something between checks.

One thing join does not do: it doesn't return the worker function's return value. Whatever target returned is discarded by the threading machinery. If the target raised an exception, the thread dies inside itself and the main thread never finds out, unless the exception is caught explicitly inside the target.

The Thread Lifecycle

A Thread object moves through three observable states: not started, running, and finished. is_alive() is the standard way to check the current state.

A thread is alive from the moment start() returns until its target function returns (or raises). Before start(), the thread hasn't been created; is_alive() returns False. After the target finishes, is_alive() returns False again. The Thread object itself sticks around, its attributes can still be read, but it can't be restarted.

A thread enters Running exactly once and leaves it exactly once. There's no Paused, no Suspended, no externally-triggered termination. Python threads run cooperatively until their target finishes or raises.

Every thread has an OS-level integer ID once it starts. The ident attribute exposes it, or threading.get_ident() from inside the thread:

Before start, the thread doesn't exist yet, so ident is None. After start, the value is a non-zero integer that's unique while the thread is alive. The same value can be reused for a later thread after this one exits. There's also a native_id attribute (added in Python 3.8) that returns the OS-assigned ID, which can be handy for matching threads in external tools like top or htop.

Daemon Threads

Every thread is either a daemon or a non-daemon. The flag controls one thing: what happens when the main thread finishes.

A non-daemon thread keeps the program alive. If the main thread reaches the end of the script while a non-daemon worker is still running, Python waits for that worker before the interpreter exits. Non-daemon is the default.

A daemon thread is the opposite. When the main thread and all other non-daemon threads exit, daemon threads get killed mid-instruction. The program doesn't wait. This is useful for background tasks that should die with the program: a thread polling a metrics endpoint, a thread flushing logs, a heartbeat sender.

Without daemon=True, that while True loop would keep the program running forever. With it, the moment the main thread reaches the end of the script, the daemon is killed and the process exits cleanly.

The daemon flag can also be set through the attribute, but only before start:

Once the OS thread is created, the daemon flag is locked in. The same rule applies whether daemon=True is passed to the constructor or set on the attribute later.

Daemon threads die abruptly when the program exits. They don't get a chance to flush buffers, close files, release locks, or finish writing data. Use them for fire-and-forget background tasks. Anything that touches important external state should be a non-daemon thread with a clean shutdown path.

A practical rule: if losing the work mid-step would corrupt something a customer cares about (an order, a payment, a database row), don't make the thread a daemon. If the work is repeated periodically and missing one cycle is harmless (a stats flush, a cache refresh), daemon is fine.

Thread Names and Identifiers

Every thread has a name. By default, Python gives it something like Thread-1, Thread-2, and so on. The name keyword argument overrides that when constructing the thread, which makes log lines easier to read when several threads are running at once:

threading.current_thread() returns the Thread object for whichever thread calls it. Its .name attribute is what appears in the log line. The name attribute can be read and written directly:

In recent Python versions, the default name includes the target function's name in parentheses, which already helps when an explicit name isn't set. Setting a custom name is still worth doing for any worker that runs alongside others, because it shows up in log records that use the %(threadName)s formatter and in profiling tools.

The OS-level integer ID lives in two places. thread.ident is the Python-level identifier set by the threading module. thread.native_id, added in Python 3.8, is the ID the operating system itself assigns. The two can be different on some platforms. For matching Python threads with what top or ps shows, native_id is the right one.

AttributeWhat it isExample use
nameA string label, default Thread-N (target)Log messages, debugging
identAn integer assigned by Python, None before startInternal identification
native_idThe OS thread ID (Python 3.8+)Matching with system tools like top
daemonTrue or False, locked after startLifecycle control
is_alive()True while the thread is runningPolling for completion

Returning Values from Threads

With Thread(target=fn), whatever fn returns is thrown away. The threading machinery has no way to give it back to the caller because the caller and the thread are running concurrently and join() doesn't return values.

The workaround is to put the result somewhere the caller can find it. Three patterns cover almost every case.

Pattern 1: subclass and store on self. The thread keeps its own result as an attribute, and the caller reads it after join:

The result lives on the thread instance, so there's no shared dict to manage and no coordination between workers. Each worker writes to its own attribute.

Pattern 2: shared dict keyed by something unique. Useful with the function form when defining a class is overkill:

Each thread writes to its own key, so the writes don't collide in practice. This pattern stays safe only as long as every thread writes a different key. The moment two threads can write the same key (or one reads while another writes), a lock around the dict is required.

Pattern 3: use a queue. queue.Queue is thread-safe by design. Workers put results onto the queue and the main thread pulls them off in any order. This pattern works well when preserving the input order doesn't matter:

The queue handles all the synchronization internally. No locks to manage, no per-thread dict keys to design. The order of items reflects the order in which threads finished, not the order in which they were started.

Beyond these patterns, concurrent.futures (lesson 10) provides Future objects with .result() calls that block and return the value, just like a synchronous function. Use it for production code with many workers.

PatternBest forCaveat
Subclass + self.resultA few stateful workersMore boilerplate per worker
Shared dict, unique key per threadMany workers, function formSafe only with unique keys
queue.QueueMany workers, order doesn't matterLoses the input-to-output mapping unless you put pairs
concurrent.futures.FutureMost production codeHigher-level wrapper around the lower-level mechanisms above

Thread-Local Storage

Sometimes each thread needs its own copy of a variable, not a shared one. A common case is a per-thread database connection, an HTTP client, or a cache. threading.local is the standard way to express this.

A threading.local() instance looks like a regular object, but every attribute set on it is visible only to the thread that set it. Different threads see different values for the same attribute name:

Thread t1 sets current.customer_id to 101. Thread t2 sets the same attribute to 202 on the same current object. Neither thread overwrites the other's value, because each thread has its own dictionary of attributes inside the threading.local instance. When the main thread reads current.customer_id, it raises AttributeError, because the main thread never set anything on current.

A practical use: per-thread cached state. Consider a function that builds a fresh customer-info object the first time it's called and reuses it afterwards. If many threads call it, the goal is one cached object per thread, not one global cached object:

Each thread opens its session once and then reuses it. The two threads don't share a session, even though they share the state object. This pattern fits when each thread needs to remember something across calls without threading an extra argument through every function.

Some facts about threading.local:

  • Attributes set in one thread are not visible in another, including the main thread.
  • The first read of an attribute that was never set in this thread raises AttributeError.
  • Cleanup is automatic when the thread ends: its slice of the threading.local instance is garbage-collected along with the thread.
  • It's the wrong tool when sharing data between threads. For that, use plain variables (with locks) or a queue.

threading.local has a small per-access overhead compared to plain attribute access. It's negligible for sessions or one-time setup, but in a tight inner loop, copy the value into a local variable for the duration of the loop.

Common Patterns: Launching N Workers

The canonical shape of code that spawns several threads and waits for them all is two passes over a list: one to start each thread, one to join. Doing the start and the join in the same loop runs them serially, which defeats the purpose.

Five fetches at half a second each take half a second total instead of two and a half. Every thread is waiting at the same time, so the waits overlap.

Swapping the two loops into one (start, join, start, join, ...) serializes the work:

Each iteration starts a thread and immediately waits for it to finish before moving on. The total time becomes the sum of the per-thread times, not the maximum. The fix is to collect the threads first and join in a separate pass.

Common Patterns: Fire-and-Forget

A fire-and-forget thread is one that's started without any wait on completion. The classic case is sending a metric, writing a log line, or kicking off a side effect that doesn't block the main flow.

The thread is created, started, and never joined. Marking it as a daemon means it won't keep the program alive if the main thread exits before the logging finishes. The time.sleep(0.2) in the example exists only so the daemons have a chance to print before the program exits; a long-running service wouldn't need it.

Two warnings about this pattern:

  1. Daemons die abruptly. If the log write needs to actually finish, this pattern can lose entries when the program exits. For important work, use a non-daemon thread with a clean shutdown path, or push to a queue that a dedicated worker drains.
  2. Spawning a fresh thread for every event is wasteful at scale. Past a few events per second, a single background worker pulling from a queue.Queue is more efficient than a thread-per-event.

For low-volume background side effects (a few per second at most), fire-and-forget daemon threads are fine. For heavier loads, use thread pools or proper task queues.

Putting It Together

Creating a thread in Python comes down to a small set of moving parts. Build a Thread instance with a target function or by subclassing. Call start() once to launch it, join() to wait for it, and is_alive() to check if it's still running. Set daemon=True for background tasks that should die with the program. Name the thread for log readability. Get values back through attributes, shared dicts, or queues, because join doesn't return them. Use threading.local when each thread needs its own private state.

These are the mechanics of creating threads.

Quiz

Creating Threads Quiz

10 quizzes