defaultdict is a dictionary subclass from the collections module that removes the "check if the key exists, and if not, create an empty value" boilerplate that shows up every time you group, count, or index data. You tell it what the default should be, and it builds the entry for you the first time a missing key is touched. This lesson covers the factory functions, the common patterns, and the pitfalls.

Picture this. You have a list of orders, each tagged with a customer ID, and you want to group them so you can see every order a given customer placed. With a plain dict, the natural code looks like this:

The logic works, but those three lines in the middle (if customer not in ..., create the list, append) are pure boilerplate. The interesting work is just append. Every grouping, counting, or indexing job repeats this dance.

dict.setdefault is the first cleanup you might reach for:

Better. The if is gone. But setdefault has a subtle cost: the default value ([] here) is constructed on every call, whether the key is missing or not. For a list it's cheap, but if the default were something heavier, you'd be allocating a new object on every iteration just to throw it away.

defaultdict removes both the boilerplate and the wasted construction:

The grouping loop is now one line of logic. When orders_by_customer["alice"] is accessed for the first time, defaultdict calls list(), stores the empty list under "alice", and returns it. The .append("Wireless Mouse") then runs on that fresh list. The second time orders_by_customer["alice"] is touched, the list is already there, so no construction happens.

defaultdict is a subclass of dict. Everything you know about dictionaries (get, pop, keys, values, iteration, comprehensions) carries over. The only thing it adds is a hook for missing keys.

You construct it with a factory: a zero-argument callable that produces the default value. list, int, set, dict, and any custom function or lambda all qualify.

Three things to notice. First, cart_counts["wireless mouse"] didn't raise KeyError even though the key was never set. Second, it returned 0, which is what int() produces. Third, the key now exists in the dictionary. defaultdict didn't just give you a default value on the fly; it actually inserted the entry.

The flow on the right (the orange "no" branch) is the whole reason defaultdict exists. It's a small piece of machinery, but it's the difference between writing the grouping loop in three lines and writing it in one.

The factory is called with no arguments. That's why you pass list and not list(). You're handing defaultdict the function itself, and defaultdict calls it later, only when it needs to.

The error fires the moment you access a missing key, not at construction. The fix is to drop the parentheses: the factory has to be callable.

Grouping is the single most common use of defaultdict. Anywhere you have records tagged by a category and want a list of records per category, defaultdict(list) is the right tool.

The first time a category is seen, defaultdict calls list() to build an empty list and stores it under that category key. From that point on, the key exists and append just adds to the existing list. The whole grouping is one loop body line.

This pattern works for any "many records per key" shape. Group orders by customer, reviews by product, employees by department, deliveries by date, refunds by reason. The factory stays list, only the iteration changes.

The second most common pattern is counting. If the factory is int, the default value is 0 (because int() returns 0), and += on a missing key works out naturally.

When quantities["Wireless Mouse"] is touched for the first time, the factory produces 0, the entry is inserted, and += 1 makes it 1. Subsequent appearances of "Wireless Mouse" find the existing entry and bump it.

This works for any aggregation that starts at zero: totals, counts, running sums. The factory int is doing more than counting; it's shorthand for "the additive identity, which is 0".

Same shape, different factory (float instead of int), different operation (+= amount instead of += 1). Both rely on the same idea: the factory produces the identity value for the operation you're going to perform.

A note on Counter

If you're doing pure counting (incrementing by 1 per occurrence), the standard library has a more specialized tool: collections.Counter. It's a defaultdict(int) cousin with extras like .most_common(n), addition/subtraction of counts, and built-in handling of any iterable.

For straight counting, prefer Counter. Reach for defaultdict(int) when you're incrementing by varying amounts (totals, weighted sums) or when you want a single structure that mixes counts with other operations.

Sometimes you want to group items, but with duplicates removed and membership tests in O(1). That's what set is for, and defaultdict(set) plus .add gives you a clean way to build a multi-value index.

A tag-to-products index is the textbook example. Every product can have multiple tags, and you want to look up "all products with the tag bestseller" without scanning the whole catalog.

Two things are happening that set does for you. First, if the same (product, tag) pair somehow appears twice in the source data, the set guarantees the product only shows up once under that tag. Second, lookups like "Wireless Mouse" in products_by_tag["bestseller"] are O(1) instead of O(n) with a list. For an index that's queried often, that matters.

The reverse index (product to tags) is the same pattern with the roles swapped:

set.update(iterable) is the bulk version of add; it's the right call when the value coming in is already a collection.

Sometimes the default needs to be something other than the five built-in factory types. Maybe you want defaultdict(int) nested inside another defaultdict, so you can do counts[customer][product] += 1 without thinking about either level. A lambda lets you build any zero-argument factory.

The outer factory is lambda: defaultdict(int). Each time a missing customer is touched, the lambda runs and produces a fresh inner defaultdict(int). That inner dict in turn defaults to 0 for any missing product. Two layers of "auto-create on access", composed cleanly.

Why a lambda? Because defaultdict(defaultdict(int)) would call defaultdict(int) once, at construction time, and share that same inner dictionary across every outer key. That's almost never what you want. The lambda defers construction, so each outer key gets its own inner dictionary.

Both customers see the same count, because both are reading and writing the same inner dictionary. The lambda form is the fix.

Lambdas also handle other custom defaults: a list of zeros, a tuple with sensible starting values, or any small object.

The lambda creates a fresh four-zero list for each new customer. Each customer's counts are independent because each access produces a new list.

Both defaultdict and dict.setdefault solve the same surface problem: returning a default when a key is missing. The differences are in when the default is constructed and what gets inserted.

The "expensive default" line is the one that bites people. Consider:

Only one "building empty report..." line, because the factory ran only on the first access. The next two reads found the existing entry.

Compare to setdefault:

The make_empty_report() call evaluates before Python calls setdefault, so it runs every time, even though only the first call's result is actually inserted. The other two reports are built and immediately discarded.

For cheap defaults like [] or 0, this doesn't matter. For expensive ones, it does. setdefault is still useful when you need the lazy-create behavior at one or two spots and want to keep the type as a plain dict. For a loop that does grouping or counting, defaultdict wins.

This is the part most people get wrong on their first defaultdict bug. Reading a missing key inserts it. Not just d[k] = ..., but also d[k] and d[k].append(...) and even if d[k]:.

"HDMI Cable" is now in the dictionary with a value of 0, even though it was never explicitly added. The if quantities["HDMI Cable"]: looked innocent, but the access went through defaultdict's missing-key machinery, which both produced the default and inserted it. If you later iterate the keys to find products that were actually ordered, you'll see ghosts.

To safely check for a key without inserting, use in or get:

in returns a bool without touching the missing-key path. get returns None (or the default you pass) without inserting either.

The rule of thumb: only use bracket notation (`d[key]`) when you actually want the auto-create behavior. For any read that should be free of side effects, use get or in. Once that distinction is internal, the rest of the defaultdict API is simple.

Once you've finished building a defaultdict, you often want to freeze it into a plain dict. Reasons include: serializing to JSON (which doesn't care, but plain dicts are easier to reason about), passing to code that expects exactly dict, or just preventing accidental auto-create from any later reader.

dict(some_defaultdict) is a shallow conversion. It copies the top-level mapping into a new plain dict, but the values (lists, sets, inner dicts) are the same objects. For a single-level defaultdict, that's all you need. For a nested defaultdict, you have to convert each level.

The dict comprehension is the standard fix for nested cases. There's no built-in "deep convert" for defaultdict; you write the one level of recursion you need.

A common alternative if you want to keep the same object but disable auto-create from now on, is to set default_factory to None. The next section covers that.

The factory you passed to the constructor is stored on the instance as default_factory. You can read it, change it, or disable it entirely by setting it to None.

Setting default_factory = None is the "freeze" knob. The object stays a defaultdict, but reads of missing keys now behave like a plain dict: KeyError. This is a way to switch from "build phase" to "read phase" within the same object, without converting to a plain dict.

You can also swap factories partway through, although doing so is unusual and can confuse readers. If you find yourself reaching for that, it's often a sign you want two different dictionaries.

Now alice's value is a list (created before the swap), and bob's value is a set (created after). The container types are mixed, which is exactly the kind of inconsistency that breaks downstream code expecting one shape. Use this sparingly.

The most useful pattern with default_factory is the freeze: build with auto-create, then set the factory to None to lock the dictionary against accidental new keys for the rest of its life.

A few patterns combine the pieces above into the shapes you'll see in actual code.

The grouping is one line; the aggregation that follows is just plain Python over the grouped data. This separation is the whole appeal: defaultdict does the dull part, your loop does the interesting part.

The defaultdict(int) handles both adds and removes uniformly. The cleanup step (dict comprehension) drops any item whose net quantity ended at zero.

Alice reviewed the mouse twice in the source data, but the set keeps a single entry per reviewer per product. If you'd used defaultdict(list), you'd have to deduplicate afterwards. The factory choice does the work for you.

The two-level defaultdict keeps the loop body short. Converting inner dicts at print time gives clean output without changing the underlying structure.