{"title":"functools Module","description":"","content":"`functools` is the standard library's collection of tools for working with functions. Most of what it provides are decorators and helpers that could be written by hand, but the module bundles the well-tested versions. This lesson covers the parts most commonly used: `@cache` and `@lru_cache` for memoization, `@wraps` for preserving function metadata, `partial` for binding arguments ahead of time, `reduce` (briefly), `@cached_property` for per-instance lazy attributes, `@singledispatch` for dispatching on argument type, and `@total_ordering` for filling in the missing comparison methods on a class. Each solves a specific problem; together they cover most of the patterns that come up when functions need extra structure.\n\n---\n\n# Memoization with `@cache` and `@lru_cache`\n\nA function that does expensive work for the same inputs every time wastes effort. Memoization is the technique of storing each result so repeat calls return the cached value instead of recomputing. `functools` ships two decorators for this.\n\nThe simpler one is `@cache` (added in Python 3.9). It caches every unique argument combination with no limit.\n\n\n```python\nfrom functools import cache\n\n@cache\ndef shipping_cost(weight_kg, zone):\n print(f\"computing for {weight_kg}kg to zone {zone}\")\n return round(5.00 + weight_kg * 0.50 + zone * 1.25, 2)\n\nprint(shipping_cost(2.5, 3))\nprint(shipping_cost(2.5, 3))\nprint(shipping_cost(1.0, 1))\nprint(shipping_cost(2.5, 3))\n```\n\n\nThe \"computing\" message prints only for unique argument combinations. The first call to `shipping_cost(2.5, 3)` computes and stores. The second and fourth calls with the same arguments return the cached value without re-running the body. The third call has different arguments, so it computes a new result and stores that too.\n\nThe older and more configurable version is `@lru_cache(maxsize=N)`. LRU stands for \"least recently used\", which is the eviction strategy: when the cache fills up, the entry that was used least recently gets discarded to make room for a new one.\n\n\n```python\nfrom functools import lru_cache\n\n@lru_cache(maxsize=128)\ndef shipping_cost(weight_kg, zone):\n return round(5.00 + weight_kg * 0.50 + zone * 1.25, 2)\n\nprint(shipping_cost(2.5, 3))\nprint(shipping_cost(2.5, 3))\nprint(shipping_cost.cache_info())\n```\n\n\n`cache_info()` is a method that any `lru_cache`-decorated function provides. It returns a named tuple with four fields: how many calls hit the cache, how many missed, the maximum size, and how many entries are currently stored. This is the standard way to check whether the cache is paying off; a cache with zero hits is overhead.\n\n\n```python\nfrom functools import lru_cache\n\n@lru_cache(maxsize=128)\ndef fibonacci(n):\n if n < 2:\n return n\n return fibonacci(n - 1) + fibonacci(n - 2)\n\nprint(fibonacci(30))\nprint(fibonacci.cache_info())\n```\n\n\nWithout the cache, `fibonacci(30)` would make over two million recursive calls because of the overlapping subproblems. With the cache, each unique `n` is computed once and reused, bringing the call count down to one per value. The hit/miss ratio tells the whole story.\n\nThe cache can be cleared manually with `cache_clear()`, which is useful in tests or when underlying data has changed:\n\n\n```python\nfrom functools import lru_cache\n\n@lru_cache(maxsize=128)\ndef lookup(product_id):\n return product_id.upper()\n\nlookup(\"sku-1\")\nlookup(\"sku-2\")\nprint(lookup.cache_info())\n\nlookup.cache_clear()\nprint(lookup.cache_info())\n```\n\n\nThe difference between `@cache` and `@lru_cache` is the `maxsize`. `@cache` is `@lru_cache(maxsize=None)`: unbounded, never evicts. `@lru_cache` with a number sets a ceiling and evicts the least recently used entry when the ceiling is reached. For most cases, `@cache` is fine; use `@lru_cache(maxsize=...)` when the input space is large enough that an unbounded cache would grow without limit.\n\nOne important constraint: **arguments have to be hashable**. The cache uses the arguments as dict keys, so lists, dicts, and sets don't work directly. Tuples, strings, numbers, and frozensets are fine.\n\n\n```python\nfrom functools import cache\n\n@cache\ndef cart_total(prices):\n return sum(prices)\n\n# This works because the argument is a tuple.\nprint(cart_total((29.99, 14.99, 9.99)))\n\n# This fails because lists aren't hashable.\nprint(cart_total([29.99, 14.99, 9.99]))\n```\n\n\nThe error message is the standard Python \"unhashable type\" complaint. The fix is to pass a tuple (`(29.99, 14.99, 9.99)`) instead of a list, or to redesign the function to accept individual arguments with `*args`. To cache a function that takes a list, convert it: `cart_total(tuple(prices))`.\n\nCache lookups are O(1) on the size of the cache (it's a dict internally). The cost per call is one hash and one dict lookup, plus the overhead of building the cache key from the arguments. For functions that do non-trivial work, the cache is essentially free. For trivial functions, the cache can be slower than running the body, because the hash and lookup cost more than the function. Profile before assuming the cache helps.\n\n---\n\n# `@wraps`: Preserving Function Identity\n\nThe decorators lesson introduced `@wraps`. It's in `functools` because it's a tool for writing decorators, but a quick recap here covers it for completeness.\n\nA decorator wraps a function in another function. Without help, the wrapper takes over the wrapped function's identity: its name, its docstring, its module, its qualified name. `@wraps(func)` copies those attributes from `func` onto the wrapper, so the result looks like the original from any introspection tool.\n\n\n```python\nfrom functools import wraps\n\ndef announce(func):\n @wraps(func)\n def wrapper(*args, **kwargs):\n print(f\"calling {func.__name__}\")\n return func(*args, **kwargs)\n return wrapper\n\n@announce\ndef cart_total(prices):\n \"\"\"Sum the prices in the cart.\"\"\"\n return sum(prices)\n\nprint(cart_total.__name__)\nprint(cart_total.__doc__)\nprint(cart_total.__wrapped__)\n```\n\n\nThe first two lines confirm that the wrapped function looks like the original. The third line is a useful detail: `@wraps` also sets `__wrapped__` on the wrapper, which points back to the original undecorated function. That's the way to call past the wrapper when needed (in tests, for example, or to bypass a cache as in the quick check above).\n\nAlways use `@wraps` inside any decorator. The decorators lesson covers the rule and the consequences of forgetting; `functools.wraps` is the tool that does it.\n\n---\n\n# `partial`: Binding Arguments Ahead of Time\n\n`functools.partial` creates a new function from an existing one with some of its arguments already filled in. The new function needs only the remaining arguments. It's the standard way to specialize a general-purpose function for a specific use without writing a whole new function definition.\n\n\n```python\nfrom functools import partial\n\ndef shipping_cost(weight_kg, zone, express=False):\n base = 5.00 + weight_kg * 0.50 + zone * 1.25\n if express:\n base += 10.00\n return round(base, 2)\n\n# Build specialized versions.\nzone_1_shipping = partial(shipping_cost, zone=1)\nexpress_zone_3 = partial(shipping_cost, zone=3, express=True)\n\nprint(zone_1_shipping(2.5))\nprint(express_zone_3(2.5))\nprint(express_zone_3(weight_kg=1.0))\n```\n\n\n`partial(shipping_cost, zone=1)` returns a new callable that behaves like `shipping_cost` with `zone=1` already supplied. Calling `zone_1_shipping(2.5)` is equivalent to calling `shipping_cost(2.5, zone=1)`. The same pattern works for any combination of positional and keyword arguments; the partial captures whatever it's given.\n\nPositional arguments can also be bound:\n\n\n```python\nfrom functools import partial\n\ndef price_with_tax(price, tax_rate, discount=0.0):\n return round(price * (1 - discount) * (1 + tax_rate), 2)\n\n# Bind the tax rate by keyword to be explicit.\nca_price = partial(price_with_tax, tax_rate=0.0725)\nny_price = partial(price_with_tax, tax_rate=0.04)\n\nprint(ca_price(100))\nprint(ny_price(100, discount=0.1))\n```\n\n\nTwo specialized pricing functions built from one general one. The original `price_with_tax` is unchanged; `ca_price` and `ny_price` are new callables that supply different tax rates by default.\n\nThe classic use case for `partial` is **callbacks and higher-order functions**: passing a function to another function (a sort key, a button handler, a map callback) when the target function needs extra arguments the receiver doesn't know about.\n\n\n```python\nfrom functools import partial\n\nproducts = [\n {\"name\": \"Wireless Mouse\", \"price\": 29.99, \"rating\": 4.5},\n {\"name\": \"USB Cable\", \"price\": 4.99, \"rating\": 4.8},\n {\"name\": \"HDMI Cable\", \"price\": 14.99, \"rating\": 4.6},\n {\"name\": \"Webcam\", \"price\": 89.99, \"rating\": 4.2},\n]\n\ndef sort_key(product, by):\n return product[by]\n\n# Sort by price.\nby_price = sorted(products, key=partial(sort_key, by=\"price\"))\nfor p in by_price:\n print(f\"{p['name']}: ${p['price']}\")\n\nprint()\n\n# Sort by rating.\nby_rating = sorted(products, key=partial(sort_key, by=\"rating\"), reverse=True)\nfor p in by_rating:\n print(f\"{p['name']}: {p['rating']}\")\n```\n\n\n`sorted` expects `key` to be a one-argument function that takes a product and returns the value to sort on. `sort_key` takes two arguments. `partial(sort_key, by=\"price\")` produces a one-argument function with `by=\"price\"` baked in, which is what `sorted` needs.\n\nThe same can be done with a lambda (`key=lambda p: p[\"price\"]`) or with the standard `operator.itemgetter(\"price\")`. For a single use, the lambda is shorter. For a reusable pattern across many calls, `partial` keeps the function named and reusable.\n\nA `partial` object is callable like any function, with three useful attributes: `.func` is the wrapped function, `.args` is the tuple of bound positional arguments, and `.keywords` is the dict of bound keyword arguments.\n\n\n```python\nfrom functools import partial\n\ndef greet(greeting, name, punctuation=\"!\"):\n return f\"{greeting}, {name}{punctuation}\"\n\nhello = partial(greet, \"Hello\", punctuation=\".\")\nprint(hello.func)\nprint(hello.args)\nprint(hello.keywords)\nprint(hello(\"alice\"))\n```\n\n\nThe introspection attributes are mostly useful for debugging and for code that needs to know what arguments are already bound. Day-to-day, the partial is called like a function and the internals don't matter.\n\n---\n\n# `reduce`: Folding a Sequence\n\n`functools.reduce` repeatedly applies a two-argument function to a sequence, accumulating a single result. It's the standard \"fold\" operation from functional programming.\n\nHere's the brief version. The shape is `reduce(func, iterable, initial=...)`: it takes the first two elements (or the initial value plus the first element), applies `func` to them, then applies `func` to that result and the next element, and so on.\n\n\n```python\nfrom functools import reduce\n\nprices = [29.99, 14.99, 9.99, 49.99]\n\n# Sum prices: equivalent to sum(prices).\ntotal = reduce(lambda a, b: a + b, prices)\nprint(total)\n\n# Find max price: equivalent to max(prices).\nbiggest = reduce(lambda a, b: a if a > b else b, prices)\nprint(biggest)\n\n# With an initial value.\ntotal_with_tax = reduce(lambda a, b: a + b, prices, 100)\nprint(total_with_tax)\n```\n\n\nMost of what `reduce` can do is better expressed with the built-in `sum`, `max`, `min`, or `any`/`all`. `reduce` fits when the operation isn't covered by a built-in and isn't a simple comprehension: combining nested dicts, building a custom accumulator, applying a chain of transformations.\n\n\n```python\nfrom functools import reduce\n\n# Merge a list of dicts, later keys winning.\nconfigs = [\n {\"timeout\": 10, \"retries\": 3},\n {\"timeout\": 30, \"log_level\": \"INFO\"},\n {\"retries\": 5, \"log_level\": \"DEBUG\"},\n]\n\nmerged = reduce(lambda a, b: {**a, **b}, configs)\nprint(merged)\n```\n\n\nThree dicts merged into one, with each later dict overwriting earlier keys. `reduce` runs the merge function pairwise across the list.\n\n---\n\n# `@cached_property`: Lazy Instance Attributes\n\n`@cached_property` (added in Python 3.8) is like `@property` but the result is computed once and stored on the instance, so subsequent accesses return the cached value without re-running the method. Use it when an instance has an expensive-to-compute value that doesn't change for that instance's lifetime.\n\n\n```python\nfrom functools import cached_property\n\nclass Cart:\n def __init__(self, items):\n self.items = items\n\n @cached_property\n def total(self):\n print(f\"computing total for cart with {len(self.items)} items\")\n return sum(item[\"price\"] for item in self.items)\n\ncart = Cart([\n {\"name\": \"Wireless Mouse\", \"price\": 29.99},\n {\"name\": \"USB Cable\", \"price\": 4.99},\n {\"name\": \"HDMI Cable\", \"price\": 14.99},\n])\n\nprint(cart.total)\nprint(cart.total)\nprint(cart.total)\n```\n\n\nThe \"computing\" line prints once. The first access runs the method and stores the result as an attribute on `cart`. The second and third accesses read the attribute. Compared to plain `@property`, which would re-run the method on every access, `@cached_property` trades a small amount of memory (one attribute per instance) for one-time computation.\n\nThe cache lives on the instance, not on the class. Two different `Cart` instances each compute their own total once:\n\n\n```python\nfrom functools import cached_property\n\nclass Cart:\n def __init__(self, items):\n self.items = items\n\n @cached_property\n def total(self):\n print(f\"computing total\")\n return sum(item[\"price\"] for item in self.items)\n\na = Cart([{\"name\": \"Mouse\", \"price\": 29.99}])\nb = Cart([{\"name\": \"Cable\", \"price\": 4.99}])\n\nprint(a.total)\nprint(b.total)\nprint(a.total)\nprint(b.total)\n```\n\n\nEach instance computes its total exactly once. The two caches are independent.\n\nOne catch: because the cache is stored on the instance, the instance has to allow attribute assignment. Classes that use `__slots__` to restrict attributes can't use `@cached_property` unless `__slots__` includes the property name explicitly. That's a niche concern for most code, but the error message (\"object has no attribute\") looks unrelated to the cache.\n\nTo invalidate the cache, delete the attribute:\n\n\n```python\nfrom functools import cached_property\n\nclass Cart:\n def __init__(self, items):\n self.items = items\n\n @cached_property\n def total(self):\n print(\"computing total\")\n return sum(item[\"price\"] for item in self.items)\n\ncart = Cart([{\"name\": \"Mouse\", \"price\": 29.99}])\nprint(cart.total)\nprint(cart.total)\n\n# Items changed; invalidate the cache.\ncart.items.append({\"name\": \"Cable\", \"price\": 4.99})\ndel cart.total\nprint(cart.total)\n```\n\n\nThe `del cart.total` removes the cached attribute. The next access re-runs the method and stores the new result. Forgetting to invalidate after mutating the underlying data leaves the cached value stale, the classic caching bug.\n\nFirst access does the full computation plus one attribute assignment. Subsequent accesses are one attribute read, which is among the fastest operations Python has. The memory cost is one attribute per instance per cached property. For a cart with a few cached properties, this is invisible.\n\n\n```mermaid\nflowchart LR\n First[\"cart.total
(first access)\"]:::cyan\n Method[\"method runs
computes value\"]:::orange\n Store[\"value stored
on instance\"]:::green\n Second[\"cart.total
(later access)\"]:::cyan\n Read[\"attribute read
(no method call)\"]:::teal\n\n First --> Method --> Store\n Second --> Read\n\n classDef cyan fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\nThe diagram contrasts the two paths. The first access runs the method and stores the result. Every later access skips the method entirely and reads the stored attribute. The cache lives in the same dict that holds every other instance attribute, so reads are fast.\n\n---\n\n# `@singledispatch`: Function Overloading by Type\n\nPython doesn't have function overloading the way some other languages do; `def add(int, int)` and `def add(str, str)` can't coexist as two separate functions in the same scope. `@singledispatch` is the standard-library workaround: it allows registering multiple implementations of a function, one per argument type, and Python picks the right one at call time based on the type of the first argument.\n\n\n```python\nfrom functools import singledispatch\n\n@singledispatch\ndef format_value(value):\n return f\"unknown: {value!r}\"\n\n@format_value.register(int)\ndef _(value):\n return f\"int: {value:d}\"\n\n@format_value.register(float)\ndef _(value):\n return f\"price: ${value:.2f}\"\n\n@format_value.register(str)\ndef _(value):\n return f\"text: {value!r}\"\n\n@format_value.register(list)\ndef _(value):\n return f\"list of {len(value)} items\"\n\nprint(format_value(42))\nprint(format_value(29.99))\nprint(format_value(\"Wireless Mouse\"))\nprint(format_value([\"Mouse\", \"Cable\"]))\nprint(format_value({\"name\": \"Alice\"}))\n```\n\n\nThe `@singledispatch` decorator turns `format_value` into a dispatcher. The base implementation (the one decorated with `@singledispatch` itself) handles any type that doesn't have a more specific registration. Each `@format_value.register(SomeType)` registers a specialized implementation for that type. At call time, Python looks at the type of the first argument and picks the matching implementation.\n\nThe convention is to name the registered implementations `_` because the name isn't used (Python looks them up through the dispatcher, not by name). Some codebases use descriptive names instead; both work.\n\nSubclasses inherit the registration. A registration for `list` doesn't match a `tuple` (it falls through to the base), but any custom class that inherits from `list` matches the `list` registration.\n\n\n```python\nfrom functools import singledispatch\n\n@singledispatch\ndef describe(value):\n return f\"generic: {value}\"\n\n@describe.register(list)\ndef _(value):\n return f\"list with {len(value)} items\"\n\nclass FancyList(list):\n pass\n\nprint(describe([1, 2, 3]))\nprint(describe(FancyList([1, 2, 3])))\nprint(describe((1, 2, 3)))\n```\n\n\n`FancyList` is a subclass of `list`, so it matches the `list` registration. The tuple doesn't match any registered type, so it falls through to the generic implementation.\n\n`@singledispatch` only dispatches on the **first** argument. For dispatch on multiple types, a different tool is needed (multipledispatch is a third-party library, or a custom dispatcher). Most cases that look like they need multiple dispatch can be reformulated to dispatch on one type.\n\nA common use case is serialization: converting different Python objects to a JSON-friendly form, where each type has its own conversion rule.\n\n\n```python\nfrom functools import singledispatch\nfrom datetime import datetime\n\n@singledispatch\ndef to_json(value):\n raise TypeError(f\"can't serialize {type(value).__name__}\")\n\n@to_json.register(int)\n@to_json.register(float)\n@to_json.register(str)\n@to_json.register(bool)\ndef _(value):\n return value\n\n@to_json.register(datetime)\ndef _(value):\n return value.isoformat()\n\n@to_json.register(list)\ndef _(value):\n return [to_json(item) for item in value]\n\n@to_json.register(dict)\ndef _(value):\n return {k: to_json(v) for k, v in value.items()}\n\norder = {\n \"id\": \"ORD-42\",\n \"placed_at\": datetime(2024, 6, 1, 10, 30),\n \"items\": [\"Wireless Mouse\", \"USB Cable\"],\n \"total\": 34.98,\n}\nprint(to_json(order))\n```\n\n\nEach registration handles one type. The list and dict implementations recurse through their contents, calling `to_json` on each element so the dispatcher picks the right rule per item. The base function raises an error for any type that doesn't have a rule, which is a useful default: failing fast beats serializing something incorrectly.\n\nDispatch is roughly one MRO walk per call, which is fast but not free. For a function called once per request, the cost is invisible. For a function called inside a tight loop over heterogeneous data, dispatch can add up; benchmark before assuming `@singledispatch` fits hot paths.\n\n---\n\n# `@total_ordering`: Filling in Comparison Methods\n\nA class that defines `__eq__` and one ordering method (like `__lt__`) can get the rest of the ordering methods automatically with `@total_ordering`. The decorator inspects the class, finds the methods defined, and supplies the missing ones (`__le__`, `__gt__`, `__ge__`, and any of the others if a different one was defined).\n\nWithout `@total_ordering`, all six methods would be needed to support every comparison. With it, two methods suffice and the decorator handles the rest.\n\n\n```python\nfrom functools import total_ordering\n\n@total_ordering\nclass Product:\n def __init__(self, name, price):\n self.name = name\n self.price = price\n\n def __eq__(self, other):\n if not isinstance(other, Product):\n return NotImplemented\n return self.price == other.price\n\n def __lt__(self, other):\n if not isinstance(other, Product):\n return NotImplemented\n return self.price < other.price\n\n def __repr__(self):\n return f\"Product({self.name!r}, {self.price})\"\n\nmouse = Product(\"Wireless Mouse\", 29.99)\ncable = Product(\"USB Cable\", 4.99)\nhdmi = Product(\"HDMI Cable\", 14.99)\n\nprint(mouse > cable)\nprint(cable <= hdmi)\nprint(hdmi >= mouse)\nprint(sorted([mouse, cable, hdmi]))\n```\n\n\nThe `Product` class defines `__eq__` and `__lt__` only. `@total_ordering` fills in `__le__` (less than or equal), `__gt__` (greater than), and `__ge__` (greater than or equal) automatically, derived from the two provided. The `>`, `<=`, and `>=` operators in the example all work, and `sorted` uses the comparison methods to put the products in price order.\n\nThe trade-off is performance. The synthesized methods aren't as fast as hand-written ones, because they call `__eq__` and `__lt__` together to produce their answer. For most code this doesn't matter; for code that sorts huge lists of these objects in a tight loop, writing the methods directly is faster.\n\nThe decorator also has a documented gotcha: if `__eq__` returns `NotImplemented` for unknown types (the convention), `@total_ordering` propagates that through the synthesized methods. That's the desired behavior. Omitting the `isinstance` check in `__eq__` and accidentally returning `False` for unknown types leads to strange behavior when comparing with unrelated objects. The example above does the check correctly.\n\nFor new code that needs comparison support, `dataclasses` with `order=True` is often a better fit; it generates the methods at class definition time without the runtime indirection. Use `@total_ordering` when dataclasses aren't an option, or when a custom `__eq__` and `__lt__` doesn't fit the dataclass shape.\n\n---\n\n# Gotchas to Remember\n\nA few things go wrong when using `functools` for the first time.\n\n**Caches grow forever without `maxsize`.** `@cache` and `@lru_cache(maxsize=None)` never evict. If the input space is unbounded (per-user data, per-request IDs), the cache will grow without limit and eventually consume all available memory. Use `@lru_cache(maxsize=N)` with a sensible `N` for any function whose input space could be large.\n\n**Arguments must be hashable.** Lists, sets, and dicts can't be cache keys, so a function with one of those arguments will raise `TypeError: unhashable type` on every call. The workaround is to convert lists to tuples and dicts to frozensets of items before passing them in, or to redesign the function to take individual arguments.\n\n**Caches don't expire.** If the function's underlying data changes (a database row gets updated, a file gets rewritten), the cached value is stale until the cache is cleared or the entry is evicted. Stale cache reads are a classic source of \"but it works on my machine\" bugs.\n\n**`@cached_property` needs a writable instance.** Classes with `__slots__` that don't include the property name will reject the attribute assignment, and the cache can't store its value. Either include the property name in `__slots__` or use plain `@property` instead.\n\n**`partial` doesn't deeply specialize.** `partial(func, x=10)` doesn't make a function that has `x=10` as a default; it makes a new callable that always passes `x=10` to `func`. Overriding with a positional argument may collide if the function's parameter list lets `x` be positional.\n\n**`@singledispatch` looks at the first argument only.** For a method on a class (with `self` as the first argument), use `@singledispatchmethod` instead, which dispatches on the second argument because the first is the instance.\n\n\n```python\n# Common mistake: passing a list to a cached function.\nfrom functools import cache\n\n@cache\ndef cart_size(items):\n return len(items)\n\n# This works.\nprint(cart_size((\"Mouse\", \"Cable\")))\n\n# This crashes.\ntry:\n print(cart_size([\"Mouse\", \"Cable\"]))\nexcept TypeError as exc:\n print(f\"TypeError: {exc}\")\n```\n\n\nThe tuple works because tuples are hashable; the list doesn't because lists aren't. This is the most common cache error, and it always comes down to the same root cause: the cache uses the arguments as a dict key, and dict keys have to be hashable.","pageType":"python"}

functools Module

Get Premium