{"title":"map, filter & reduce","description":"","content":"`map`, `filter`, and `reduce` are Python's three classic higher-order functions for transforming, selecting, and collapsing data from an iterable. They predate list comprehensions and still show up in real code, especially when the operation is a single named function. This lesson covers how each one works, the lazy iterator behavior in Python 3, the multi-iterable form of `map`, when to prefer comprehensions, and how composing the three reads in practice.\n\n---\n\n# map: Transform Every Item\n\n`map(func, iterable)` calls `func` on every item in `iterable` and yields the results one at a time. It's the answer to the question \"I have a list of things, give me a list of transformed things.\"\n\nA typical use: a list of cart prices in dollars, and you want them in cents (because you decided to store everything as integers to avoid floating-point trouble).\n\n\n```python\nprices = [29.99, 14.99, 9.99, 49.99, 19.99]\n\nprices_in_cents = map(lambda p: int(p * 100), prices)\nprint(list(prices_in_cents))\n```\n\n\nTwo things to notice. First, `map` doesn't return a list, it returns a `map` object, which is a lazy iterator. Nothing actually runs until something consumes it (here, `list(...)`). Second, the function gets called once per item, with that item as its only argument.\n\n\n> **INFO**\n>\n> **Cost:** `list(map(...))` materializes every result into memory at once. For a million-row product feed, that's a million floats turning into a million ints all at once. If you only need to walk the result once, drop the `list(...)` and iterate the `map` object directly.\n\n\n`map` is at its cleanest when the transformation already exists as a named function. Suppose the catalog has product names from a sloppy import and you want them title-cased:\n\n\n```python\nnames = [\"wireless MOUSE\", \"usb cable\", \"HDMI cable\", \"WEBCAM\"]\n\ncleaned = list(map(str.title, names))\nprint(cleaned)\n```\n\n\nHere `str.title` is the function. You don't need a lambda because the method already takes a string and returns a string. This is the form where `map` reads better than a comprehension: `map(str.title, names)` is shorter and clearer than `[name.title() for name in names]`. Pick whichever your team finds easier to read; both are fine.\n\n---\n\n# map With Multiple Iterables\n\n`map` accepts more than one iterable. When you pass `n` iterables, the function must accept `n` arguments, and `map` calls it with one item from each iterable per step. It stops as soon as the shortest iterable runs out.\n\nPicture two parallel lists from a database query: unit prices and quantities. You want the line total for each row.\n\n\n```python\nunit_prices = [29.99, 14.99, 9.99, 49.99]\nquantities = [2, 5, 3, 1]\n\nline_totals = list(map(lambda price, qty: price * qty, unit_prices, quantities))\nprint(line_totals)\n```\n\n\nEach call gets a `price` and a `qty`, and you get one result per pair. If `quantities` had only three items, `map` would stop after three results without complaining, even though `unit_prices` has four. That silent truncation surprises people; if you want a loud error when the lengths don't match, use `zip(..., strict=True)` (Python 3.10+) instead.\n\nFor two-iterable transformations like this, many teams prefer `zip` plus a comprehension, because it spells out the pairing:\n\n\n```python\nline_totals = [price * qty for price, qty in zip(unit_prices, quantities)]\n```\n\n\nBoth produce the same list. Use whichever is clearer at the call site.\n\n---\n\n# filter: Keep What Matches\n\n`filter(func, iterable)` keeps items for which `func(item)` is truthy and drops the rest. Like `map`, it returns a lazy iterator, not a list.\n\nA common need: out of all products in a category, keep only the ones in stock.\n\n\n```python\nproducts = [\n {\"name\": \"Wireless Mouse\", \"stock\": 12},\n {\"name\": \"USB Cable\", \"stock\": 0},\n {\"name\": \"Charger\", \"stock\": 3},\n {\"name\": \"HDMI Cable\", \"stock\": 0},\n {\"name\": \"Webcam\", \"stock\": 7},\n]\n\nin_stock = filter(lambda p: p[\"stock\"] > 0, products)\nfor product in in_stock:\n print(product[\"name\"])\n```\n\n\nThe predicate (`lambda p: p[\"stock\"] > 0`) returns a bool for each product, and `filter` keeps the ones where the bool is true. Items where the predicate returns `False`, `None`, `0`, `\"\"`, or any other falsy value are dropped.\n\n\n> **INFO**\n>\n> **Cost:** `filter` is lazy. If you call `list(filter(...))` on a generator that yields a million records and only 10 match, the predicate still runs a million times. The lazy form helps when you stop early (\"find the first match\") because the rest never runs.\n\n\n### filter(None, iterable)\n\nThere's a special form: if the first argument is `None`, `filter` keeps items that are truthy on their own. This is the idiom for stripping empty strings, `None` values, or zeros from a list without writing a lambda.\n\n\n```python\nemails = [\"alice@example.com\", \"\", \"bob@example.com\", None, \"carol@example.com\"]\n\nvalid = list(filter(None, emails))\nprint(valid)\n```\n\n\nAnything that's \"falsy\" gets dropped. That includes empty strings, `None`, `0`, `0.0`, empty lists, and empty dicts. Use this only when \"falsy\" is exactly what you want; otherwise write the explicit predicate so the intent is obvious.\n\n---\n\n# reduce: Collapse Into a Single Value\n\n`reduce(func, iterable, initial)` lives in `functools`, not as a built-in. Guido moved it out in Python 3 because for sum, product, max, and min, the built-ins are clearer. But for \"collapse a sequence into one value using a custom operation\", `reduce` is still the cleanest tool.\n\n`reduce` walks the iterable left to right, carrying an accumulator. Each step calls `func(accumulator, item)` and the result becomes the new accumulator. After the last item, it returns the final accumulator.\n\nThe mechanic, in words: pick the first two items, combine them, then combine that with the third, then with the fourth, and so on.\n\n\n```python\nfrom functools import reduce\n\ncart_totals = [29.99, 14.99, 9.99, 49.99, 19.99]\n\ngrand_total = reduce(lambda running, price: running + price, cart_totals)\nprint(f\"Grand total: ${grand_total:.2f}\")\n```\n\n\nThis is also exactly `sum(cart_totals)`, and `sum` is what you should use in real code. `reduce` earns its keep when the combining operation isn't a built-in.\n\nHere's a more honest example. Each order has a discount expressed as a multiplier (0.9 means 10% off). Stack them by multiplying:\n\n\n```python\nfrom functools import reduce\n\ndiscounts = [0.9, 0.95, 0.8] # 10% off, then 5% off, then 20% off\n\ncombined = reduce(lambda a, b: a * b, discounts)\nprint(f\"Combined multiplier: {combined:.4f}\")\nprint(f\"$100 becomes: ${100 * combined:.2f}\")\n```\n\n\nThere's no built-in `product()` in plain Python (it's in `math.prod` from 3.8+ for numbers). For arbitrary combining operations, like merging dictionaries or chaining function calls, `reduce` is the right shape.\n\n\n```mermaid\nflowchart LR\n L[0.9]:::cyan\n M[0.95]:::cyan\n N[0.8]:::cyan\n S1[0.9 * 0.95
= 0.855]:::orange\n S2[0.855 * 0.8
= 0.684]:::green\n\n L --> S1\n M --> S1\n S1 --> S2\n N --> S2\n\n classDef cyan fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nThe diagram shows the running accumulator: combine the first two values into an intermediate result, then combine that with the next value, and so on. Each step shrinks the working set by one. After processing every item, you're left with a single value.\n\n### The initial Argument\n\n`reduce` has an optional third argument: an initial value for the accumulator. Three things change when you pass it:\n\n1. The accumulator starts at `initial` instead of the first item.\n2. `func` gets called once for every item in the iterable (instead of `n - 1` times).\n3. An empty iterable returns `initial` instead of raising `TypeError`.\n\nThat third point is the practical one. Without an initial value, `reduce` on an empty list crashes.\n\n\n```python\nfrom functools import reduce\n\nempty_cart = []\n\ntry:\n total = reduce(lambda a, b: a + b, empty_cart)\nexcept TypeError as e:\n print(f\"Error: {e}\")\n\ntotal = reduce(lambda a, b: a + b, empty_cart, 0)\nprint(f\"Total with initial value: {total}\")\n```\n\n\nFor any reduction where the input could be empty, pass an initial value. It's also how you reduce into a different type than the items themselves: start with `{}` and reduce into a dictionary, start with `[]` and reduce into a list (though for those, a loop or `dict.update()` is usually clearer).\n\n---\n\n# Lazy Iterators in Python 3\n\nIn Python 2, `map` and `filter` returned lists. In Python 3, they return lazy iterators. That change matters for three reasons.\n\n**One:** you can chain them on huge inputs without allocating intermediate lists. Filtering a 10-million-row product log and then mapping each match to a summary doesn't build a 10-million-row temporary list, it walks the records one at a time.\n\n**Two:** you can only iterate the result once. A `map` object is exhausted after one pass.\n\n\n```python\nprices = [10.0, 20.0, 30.0]\ndiscounted = map(lambda p: p * 0.9, prices)\n\nprint(list(discounted)) # first consumption\nprint(list(discounted)) # second consumption\n```\n\n\nThe second `list(...)` gets nothing because the iterator is already drained. If you need to iterate twice, materialize once with `list(...)` and reuse the list.\n\n**Three:** nothing runs until something consumes the iterator. The lambda inside `map` isn't called when you write `map(...)`, only when the result is iterated.\n\n\n```python\ndef shout(name: str) -> str:\n print(f\"processing {name}\")\n return name.upper()\n\nproducts = [\"mouse\", \"cable\"]\nresult = map(shout, products)\nprint(\"after map call, before consumption\")\n\nfor p in result:\n print(p)\n```\n\n\nThe \"processing\" lines come after \"after map call, before consumption\", which proves the function only runs when the loop pulls each value.\n\n\n```mermaid\nflowchart LR\n A[map / filter
created]:::cyan\n B[Iterator returned
no work done yet]:::orange\n C[list / for loop
pulls values]:::green\n D[Function runs
one item at a time]:::teal\n\n A --> B\n B --> C\n C --> D\n\n classDef cyan fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\nThe diagram shows the lifecycle. Creating the `map` or `filter` object is cheap and does no real work. The transformation runs only when something downstream asks for values. This is the same model as generator expressions (covered in §17 Comprehensions).\n\n---\n\n# map/filter vs Comprehensions\n\nFor most code, list and generator comprehensions (see §17) are the more Pythonic choice. They keep the transformation and the source close together, they read in roughly the order you'd say them out loud, and they don't need a `lambda` for simple expressions.\n\n\n| Operation | With map/filter | With comprehension |\n| --------- | --------------- | ------------------ |\n| Apply a function | `list(map(str.title, names))` | `[n.title() for n in names]` |\n| Multiply each item | `list(map(lambda p: p * 100, prices))` | `[p * 100 for p in prices]` |\n| Keep matching items | `list(filter(lambda p: p[\"stock\"] > 0, products))` | `[p for p in products if p[\"stock\"] > 0]` |\n| Drop falsy items | `list(filter(None, emails))` | `[e for e in emails if e]` |\n| Transform and filter | `list(map(f, filter(g, items)))` | `[f(x) for x in items if g(x)]` |\n\n\nThe opinionated answer most Python developers reach for: **use a comprehension by default. Use `map` or `filter` when the function is already a named function and there's no extra expression around it.**\n\nSo `map(str.lower, names)` is fine because `str.lower` is already a clean named callable. But `map(lambda p: p[\"price\"] * 0.9, products)` reads better as `[p[\"price\"] * 0.9 for p in products]`. The lambda adds visual noise that the comprehension doesn't have.\n\nFor lazy evaluation, both are available: `map`/`filter` are lazy by default, and the comprehension version is `(x * 2 for x in items)` (parentheses, not brackets), which is a generator expression.\n\n\n> **INFO**\n>\n> **Cost:** `[f(x) for x in items if g(x)]` is the same shape as `map(f, filter(g, items))` but typically slightly faster in CPython because it avoids the lambda overhead and the extra function call layer. If you've measured a hot loop and it's bottlenecked on iteration, the comprehension is usually the better starting point.\n\n\n---\n\n# Composing map and filter\n\nYou can stack `map` and `filter` to build a pipeline: filter the records you care about, then transform them.\n\n\n```python\nproducts = [\n {\"name\": \"Wireless Mouse\", \"price\": 29.99, \"stock\": 12},\n {\"name\": \"USB Cable\", \"price\": 4.99, \"stock\": 0},\n {\"name\": \"Charger\", \"price\": 19.99, \"stock\": 3},\n {\"name\": \"HDMI Cable\", \"price\": 8.99, \"stock\": 0},\n {\"name\": \"Webcam\", \"price\": 49.99, \"stock\": 7},\n]\n\nin_stock = filter(lambda p: p[\"stock\"] > 0, products)\ndiscounted_prices = map(lambda p: p[\"price\"] * 0.9, in_stock)\ntotal = sum(discounted_prices)\n\nprint(f\"Total of in-stock items after 10% discount: ${total:.2f}\")\n```\n\n\nRead it from inside out: take `products`, keep the ones in stock, multiply each price by 0.9, sum the result. Because both `filter` and `map` are lazy, no intermediate list is built. The whole pipeline streams one product at a time through the chain.\n\nThe same pipeline as a comprehension:\n\n\n```python\ntotal = sum(p[\"price\"] * 0.9 for p in products if p[\"stock\"] > 0)\nprint(f\"Total: ${total:.2f}\")\n```\n\n\nThat one line beats the three-line chain for almost any reader. Once your pipeline grows past two stages, comprehensions usually win on clarity. `map`/`filter` chains start to feel cluttered when there are three or more lambdas in a row.\n\nThere's a real readability cliff here. A single `map(str.lower, names)` is great. A chain of three lambdas piped through `map` and `filter` is harder to read than the equivalent comprehension or a small named function with a `for` loop. When in doubt, write it both ways and pick the one a teammate would understand faster.\n\n---\n\n# When reduce Is Genuinely the Right Tool\n\n`reduce` has the reputation of being clever when it shouldn't be. For sums, products, mins, maxes, and concatenations, the built-ins (`sum`, `math.prod`, `min`, `max`, `\"\".join`) are clearer and faster.\n\nBut reduce earns its place for **custom combining operations that aren't already a built-in**. Two common shapes:\n\n**Stacking dictionaries.** Suppose a customer profile is assembled from defaults, account settings, and per-session overrides, each as a dict. Later dicts win on conflicts.\n\n\n```python\nfrom functools import reduce\n\ndefaults = {\"theme\": \"light\", \"currency\": \"USD\", \"page_size\": 20}\naccount = {\"theme\": \"dark\", \"page_size\": 50}\nsession = {\"currency\": \"EUR\"}\n\nprofile = reduce(lambda acc, override: {**acc, **override}, [defaults, account, session])\nprint(profile)\n```\n\n\nEach step takes the running merged dict and the next override, builds a new dict with the later keys winning. This reads cleanly as \"fold these dictionaries together with right-wins-on-conflict.\" A loop would work too, but reduce makes the \"fold\" intent explicit.\n\n**Finding a running summary.** Compute the highest-value cart out of a list of carts:\n\n\n```python\nfrom functools import reduce\n\ncarts = [\n {\"id\": 1, \"total\": 124.50},\n {\"id\": 2, \"total\": 38.99},\n {\"id\": 3, \"total\": 245.00},\n {\"id\": 4, \"total\": 89.00},\n]\n\nbiggest = reduce(lambda a, b: a if a[\"total\"] > b[\"total\"] else b, carts)\nprint(biggest)\n```\n\n\nThis is also `max(carts, key=lambda c: c[\"total\"])`, which is shorter and more idiomatic. The reduce version is a good example of the trap: it works, but the built-in `max` with `key=` says exactly what's happening with half the noise.\n\nThe rule of thumb: if `sum`, `max`, `min`, `math.prod`, or `\"\".join` covers the operation, use them. Reach for `reduce` when the combining operation is custom (merging dicts, applying a chain of transformations, building a structured result).","pageType":"python"}

map, filter & reduce

Get Premium