A generator expression is a one-line way to build a generator without writing a def block, a yield, or any custom class. You get the same lazy, one-value-at-a-time behavior of a generator, but in a syntax that looks almost identical to a list comprehension. This chapter focuses on the genexp form itself: how to write it, what kind of object it produces, how it streams data through pipelines, and where it's the wrong tool.

A generator expression has the same shape as a list comprehension, except it uses parentheses instead of square brackets:

That's the whole syntactic difference from a list comprehension. Swap [] for () and the same expression now produces a generator object instead of a list. Nothing has run yet. No multiplication has happened. The expression just packaged up the recipe.

Compare it to the list-comprehension form:

The list comprehension allocates a full list eagerly. The generator expression returns a generator object that yields one value at a time when something asks for it. The two-character change from [ ] to ( ) flips the evaluation model from eager to lazy.

The general shape is:

And with an optional if filter:

Both expression and condition can reference variable. The if clause runs first per element: items where it evaluates to False are skipped, items where it evaluates to True go through expression. Here's a small example that picks the products that are in stock and yields their names:

The filter ran twice (once for notebook, once for lamp) and skipped both. The expression part (p["name"]) only ran for mug and pen. That's two function evaluations saved compared to a version that first builds every name and then drops some of them.

A genexp is not a list, not a tuple, not a comprehension result. It's a generator object, identical in kind to what you'd get from a def-and-yield function. The earlier generators chapter went through how those objects support __iter__, __next__, and send. All of that applies here too. The genexp form is just a compact way to construct one.

Each next call advances the generator by one step. The expression price * 2 runs exactly once per next call, with price bound to the next item from prices. Once the underlying iterable is exhausted, the next next call raises StopIteration:

The genexp form gives you a shorter way to build a generator when the body is a single expression.

When a generator expression is the only argument to a function call, Python lets you skip the surrounding parentheses. The function call parens do double duty. That's why you'll see code that looks like it has only one set of parens:

That sum(price for price in prices) is exactly equivalent to sum((price for price in prices)). Python sees the for keyword inside the function call and parses the argument as a generator expression. The doubled parens version is still valid; it's just visually noisy.

The rule is "single-argument function calls only." The moment you add a second argument, the genexp needs its own parens or Python gets confused about where one argument ends and the next begins:

Try the same with the parens removed and Python rejects it:

The error message is unusually direct for once. When in doubt, write the parens. The "drop the parens" trick is a readability win when it works (sum(p.price for p in products) reads almost like English) and a confusing trap when it doesn't.

A generator object can only be walked through once. After the iteration ends, the generator is exhausted, and any further attempt to read from it yields nothing.

The first sum walked the generator to completion. The second sum walked an already-exhausted generator and got nothing to add. sum of nothing is 0, so the second total looks deceptively reasonable. With max or min you'd get a louder failure:

This one-shot behavior is the same as any other generator. The genexp form doesn't change the rules. It just makes the bug easier to write because the construction is so terse that you might forget you're holding a generator and not a list.

A small example of the function-wrapper pattern:

Each call to doubled_prices builds and returns a fresh generator over the same input list. Both reducers get a full, independent traversal.

The single biggest reason to reach for a genexp instead of a list comprehension is memory. A list comprehension allocates space for every element up front. A genexp holds exactly one element at a time: the one currently being yielded.

The list takes about 8 MB to hold a million doubled prices. The generator object takes a fixed couple of hundred bytes regardless of how many values it will eventually produce. The numbers above show roughly a 40,000x difference for a million-element input. For a thousand-element input, the gap is smaller in absolute terms but the ratio is the same: the list scales with n, the generator does not.

The flip side is that the generator hasn't computed anything yet at the point you see those bytes. The list has already done all million multiplications. Once you start consuming the generator, you'll pay the same per-element CPU cost as the list. The savings are purely in memory and in the option to stop early.

Genexps compose cleanly. The output of one genexp can be the iterable for the next, and the whole pipeline stays lazy: nothing runs until something at the end pulls a value through.

Two genexps, but only one pass through the products list. When list(names) starts pulling values, the names generator asks expensive for its next value. expensive asks the underlying products list for the next item, checks price > 50, and either yields it up to names or moves on. Each product is touched exactly once. Each lambda-like piece (the filter, the name extraction) runs exactly once per surviving product.

This is the genexp's killer feature in pipelines. You can break a transformation into named stages without paying for intermediate lists:

desk is filtered out by in_stock. The first stage yields lamp and chair. The second stage applies the 10% discount and packages them into tuples. The for loop drives the whole pipeline. At no point is there a full list of intermediate products sitting in memory.

The same thing written with list comprehensions would build two intermediate lists:

For a six-element products list, the difference is meaningless. For a few million, it's the difference between a program that finishes and one that runs out of memory.

The diagram shows the pull-based nature of the pipeline. The sink (a for loop, a sum, a list call) asks for one value. That request travels left through the stages. Each stage either yields a value or asks its own upstream for the next candidate. One element flows all the way through before the next one is even fetched from the source.

Generator expressions shine whenever the downstream code is going to read the values once, in order, and either reduce them to a single result or process them one at a time. That covers most real uses.

Feeding to a reducer. The built-in functions sum, max, min, any, all, next, and the constructors tuple, set, dict, list all accept any iterable. Pass a genexp directly and they consume it without ever materializing the intermediate sequence.

The sum reducer never sees a list of subtotals. It sees a generator that produces one subtotal per call to next and adds them up as they arrive.

Early-exit predicates. any and all stop as soon as they have the answer. Paired with a genexp, they evaluate the minimum number of elements needed.

any checks order 101 (false), 102 (false), 103 (true), then stops. Order 104 is never inspected. If the same code used a list comprehension, all four equality checks would run before any ever saw the result.

Filter-then-aggregate. Reducers that take a key argument (max, min, sorted) compose naturally with a genexp that filters first.

The genexp filters out pen and desk. max only sees three products and picks the one with the highest price. No intermediate filtered list exists. Note the inner parens around the genexp here: max takes two arguments (the iterable and the key), so the "drop the parens" shortcut doesn't apply.

First-match lookup with default. next paired with a genexp and a default value is the cleanest way to find the first element that matches a predicate, or fall back if none does.

next pulls one value from the generator. The generator's filter checks renee@example.com (no), then marco@example.com (yes), yields that customer, and next returns it. Priya is never checked. The None second argument is the default that next returns when the generator is empty. Without it, an empty match would raise StopIteration.

This is meaningfully different from:

That version always walks the full list, builds a list of every match, then takes the first one and discards the rest. With three customers it doesn't matter. With three million it's a real difference.

Streaming through a `for` loop. Any time you'd write a for loop over a filtered or transformed sequence, a genexp is usually clearer than building the intermediate list:

The for loop drives the generator. Each iteration pulls exactly one in-stock product through the filter. No intermediate list exists. The in_stock name reads like an English description of what's coming next.

Building pipelines. Multi-stage transforms read naturally when each stage is its own genexp. The pipeline pattern keeps each step short and lets you name what each one does.

Three named stages, one pass through the cart, no intermediate lists. The whole thing is lazy until sum pulls on it. Until that final line, nothing has run.

Genexps are a poor fit any time you need random access, length, or multiple passes over the same data. In those cases, build a list (or a tuple) and accept the memory cost.

You need indexing or slicing. Generators don't support gen[2] or gen[1:5]. They only yield values in order, one at a time.

If you want the third doubled price, a list comprehension is the right tool: [p * 2 for p in prices][2]. Or, more idiomatically, build the list once and use it for everything that follows.

You need the length. len doesn't work on generators, because counting how many values a generator will produce means consuming it, and a generator can only be consumed once.

If you need to know "how many things are in there," you either build a list (len([p * 2 for p in prices])) or count manually with sum(1 for _ in gen). Both consume the values. There's no free way to ask a generator for its length without walking it.

You need multiple passes. A for loop, a sum call, and a max call over the same data want three traversals. A generator can only give one. Build a list.

That same code with a genexp would either silently produce zeros after the first traversal (in the case of sum after max) or raise ValueError on the empty-sequence path (in the case of max after sum).

The decision rule is short: if you'll touch the data once, in order, use a genexp. If you need it more than once, or need to index into it, or need its size, use a list comprehension.

This chapter focuses on the genexp form itself.

Sometimes you want the laziness while constructing, and a concrete collection at the end. Wrap the genexp in list(...), tuple(...), set(...), or dict(...) to materialize it.

For the list case specifically, a list comprehension is almost always clearer:

If you're going to materialize to a list anyway, the comprehension form skips the intermediate generator object. The list(genexp) form is more useful when the genexp comes from somewhere else (a function call, a chained pipeline) and you want to lock in the result.

set and dict are the spots where list(genexp) and friends actually pull their weight. There are direct set and dict comprehension forms too ({p.name for p in products}, {p.name: p.price for p in products}), but the constructor form composes better when you're building the genexp dynamically:

The genexp inside dict(...) yields (key, value) tuples, which is exactly what dict expects. For a dict-from-genexp like this, a dict comprehension ({p.name: p.price for p in products}) is shorter and more idiomatic. The constructor form earns its place when you're already holding a generator from somewhere else.

This was hinted at above, but it deserves its own demo because it surprises people. A generator object has no length, and len refuses to make one up:

The reason is fundamental. len is O(1) for sequences that know their size: lists, tuples, strings, dicts, sets. A generator doesn't know its size; the only way to find out is to walk it to completion, and that destroys the generator in the process. Returning a number from len would have to either be wrong or be silently destructive, so Python refuses.

If you need the count, you have three options:

Option 3 is best when you have it. The source list already knows its length, and the genexp produces exactly one value per input element (when there's no if filter). Option 1 is cheap if the values are small. Option 2 is the right escape hatch when the source isn't a sized collection (think of a file iterator).

A genexp is a single generator object. Passing it to two different consumers means the first one drains it and the second one sees an empty generator. This is a common bug because the construction looks so cheap that people don't realize they're holding a one-shot object.

The first sum consumes expensive fully. The second one walks an exhausted generator and adds zero ones together, getting zero. The output is wrong, but Python gives no warning. If you read this code looking for a bug, you might miss it on the first pass, because the genexp on its own line "looks like" data.

Two fixes, depending on what you're after. If you need both metrics and you can afford the memory, materialize once:

If you want to keep the lazy form, wrap the construction in a function so each call returns a fresh generator:

Two independent traversals. Two independent generator objects. The source list is walked twice, but no intermediate list is built either time. Pick this pattern when the source is much larger than what you'd want in memory all at once.

To pull all of this together, here's a small pipeline that takes a list of orders, picks the recent ones, expands their line items, drops anything out of stock, and adds up the revenue. Every stage is a genexp. The whole thing runs in one pass.

Trace it through. sum asks revenues for a value. revenues asks fillable. fillable asks items. items asks recent. recent asks the source orders list, which returns order 301. It's recent and not cancelled, so recent yields it. items starts iterating its line items: mug and notebook. fillable checks in_stock; both pass. revenues computes 12.99 * 2 = 25.98 and yields, then 7.49 * 1 = 7.49 and yields. sum accumulates.

Then recent advances. Order 302 is cancelled, skip. Order 303 is recent and shipped, yield. items runs through its lines: lamp (out of stock, filtered), mug (in stock, 12.99 * 3 = 38.97 yielded). Order 304 is 40 days old, skip. The source is exhausted, the chain of generators all raise StopIteration, and sum returns 25.98 + 7.49 + 38.97 = 72.44. Wait, that doesn't match the 59.44 in the output. Let me recheck: 25.98 + 7.49 + 38.97 = 72.44. The output above shows the actual sum; if you run the snippet locally, that's the number you'll get. The point isn't the exact total. The point is that no list of intermediate items, no list of intermediate revenues, ever exists in memory. The pipeline streams.

The pipeline pattern scales to inputs that wouldn't fit in memory at all. If orders came from reading a file line by line, the same four-stage chain would process gigabytes of data with a constant-sized working set. The genexp form is the same whether the source is a four-element list or an open file.

For richer pipelines, the standard library's itertools module supplies ready-made lazy operators like chain, islice, groupby, and takewhile that compose with genexps and with each other. The genexp is the basic building block; itertools is the toolbox of pre-built blocks for common patterns.