Functions are one of the main ways you keep Python AI code understandable. A good function gives one piece of work a name: call a provider, parse a document, score an answer, transform a record, or run one step of an evaluation.

Decorators and generators extend that idea. Decorators let you add timing, retry, caching, tracing, or authorization around a function without burying that infrastructure inside the function itself. Generators let you process data one item at a time, which matters when a pipeline handles thousands or millions of documents.

This chapter focuses on practical patterns you will see in API clients, ingestion jobs, model wrappers, and evaluation scripts.

In Python, functions are values. You can assign a function to a variable, store it in a list, pass it to another function, and return it from another function.

That is what "first-class" means: a function can be handled like any other object.

This matters because many Python APIs accept behavior as an argument. For example, you can sort the same evaluation results by latency or by accuracy just by passing a different key function.

sorted() does not need to understand your result schema. It only needs a function that tells it what value to sort by. First-class functions let you separate the loop from the decision being made inside the loop.

*args and **kwargs are common in AI codebases, especially in wrapper functions that forward arguments to provider SDKs, model clients, or shared utility functions.

*args collects positional arguments into a tuple. **kwargs collects keyword arguments into a dictionary. Together, they let a wrapper accept arguments without knowing the wrapped function's full signature.

The log_and_call function does not need to know what arguments embed_text accepts. It logs the call and forwards everything unchanged. That makes the wrapper less brittle when the underlying function gains another optional parameter.

A practical example is a small model-call helper with project defaults:

The {**defaults, **kwargs} pattern merges two dictionaries. Later values override earlier ones. It is a simple way to keep project defaults while still giving callers control when they need it.

A closure is a function that remembers values from the scope where it was created, even after that outer function has returned.

When make_multiplier(2) runs, it creates the inner multiply function and returns it. The returned function still has access to factor=2. That remembered value is what makes it a closure.

Here is the same idea applied to a threshold-based classifier:

Each classifier remembers its own threshold. You have created two specialized functions from one general template without introducing a class.

Closures are also the bridge to decorators, because a decorator is usually a closure that remembers the function it wraps.

Decorators are where closures become practical infrastructure.

A decorator is a function that takes another function, wraps it with extra behavior, and returns the wrapped version. The @decorator syntax is shorthand for passing a function through another function.

Decorators are easiest to understand in three steps.

From Closure to Decorator

The @timing line is equivalent to writing slow_embedding = timing(slow_embedding). The syntax is compact, but the idea is ordinary function composition.

After decoration, calling slow_embedding("hello") calls wrapper("hello"). The wrapper records the start time, calls the original function, logs the elapsed time, and returns the original result.

The @retry Decorator

Retry is one of the most useful decorator patterns in AI engineering. Remote calls fail for ordinary reasons: rate limits, timeouts, transient server errors, and network interruptions.

A retry decorator keeps that policy close to the call site without duplicating retry loops in every function:

This is a three-layer structure: retry returns decorator, and decorator returns wrapper.

That extra layer is needed because @retry(max_retries=3) has configuration. The outer function remembers the retry settings, the middle function receives the original function, and the inner wrapper runs the retry logic.

In production, retry only the errors that are actually transient, such as timeouts and temporary service failures. Be careful around side effects. Retrying a read request is usually safe. Retrying a write, payment, tool action, or database insert can create duplicates unless the operation is idempotent.

The @cache Decorator

Caching is often useful for embeddings, document parsing, lookup tables, and expensive scoring functions. If you embed the same text with the same model and dimensions twice, you may want to reuse the earlier result instead of paying for the same work again.

Python's functools module has a built-in option for simple in-process caching:

One caveat: lru_cache only works with hashable arguments. Strings, numbers, and tuples are hashable. Lists and dictionaries are not, so you cannot pass them directly to a cached function.

Also remember that lru_cache lives inside one Python process. It is not a shared cache across workers, and it disappears when the process exits. For persistent caching, distributed workers, or provider responses with privacy and retention constraints, use an explicit cache layer and decide exactly what is safe to store.

Stacking Decorators

You can apply multiple decorators to the same function. Python applies them from bottom to top, meaning the decorator closest to the function wraps it first.

In this order, timing wraps the function first, then retry wraps the timed function. That means timing measures each individual attempt. If you reversed the decorators, timing would measure the whole retry operation. The order matters.

Large AI pipelines often process more data than you want to hold in memory: raw documents, chunks, model outputs, logs, or evaluation records. If you load everything into a list, memory grows with the dataset. With a generator, you can process one item at a time.

A generator is a function that uses yield instead of returning one final value. When you call a generator function, Python does not run the body immediately. It returns a generator object. Each time you ask for the next value, either with next() or a for loop, the function runs until it reaches yield, gives back a value, and pauses. On the next iteration, it resumes from that point.

The key difference is state. A return statement ends the function. A yield statement pauses the function and keeps its local variables so it can continue later.

yield vs return

Generator Expressions

List comprehensions build lists immediately. Generator expressions produce values lazily. The syntax is almost the same, but generator expressions use parentheses instead of brackets:

The list stores every result. The generator stores only enough state to produce the next result. For large datasets, that can be the difference between steady processing and running out of memory.

Streaming LLM Responses

When you stream a model response, your application receives small chunks over time. A generator is a clean way to expose those chunks to the rest of your code:

In a real application, prefer the provider SDK's streaming helpers when they exist. With the OpenAI Python SDK, the Responses API streams typed events:

The calling code does not need to know about the provider's streaming protocol. It only iterates over text chunks.

Batching with Generators

Batching is common in AI systems. You rarely want to send one embedding request at a time if the provider supports batches. Batching can reduce request overhead and improve throughput.

Because batch_items accepts any iterable, it can consume a list, a file reader, a database cursor, or another generator. This lets you build pipelines that do not require loading the full dataset first.

Python's itertools module provides small, efficient tools for working with iterators. You do not need to know every function in the module. These three are enough to start.

chain: Flatten Multiple Iterables

chain lets you loop over multiple iterables as one continuous stream:

This is cleaner than writing three loops. It also avoids creating a new combined list.

islice: Take a Slice of Any Iterator

islice lets you take a slice from any iterator. It is useful when you want to test a pipeline on the first few items without consuming everything.

Generators do not support normal indexing or slicing. islice gives you that behavior without turning the whole iterator into a list.

batched: Native Batching (Python 3.12+)

Starting in Python 3.12, itertools includes batched, which does the same basic job as our earlier batch_items generator:

If you are on an older Python version, use the manual batch_items generator from the previous section or install the more-itertools package.

A lambda is a small anonymous function written as a single expression. Lambdas are most useful when a named function would add noise, such as a short key function for sorting.

Keep lambdas short. If the expression needs explanation, use a named function. In evaluation and production pipelines, readability matters more than saving a line of code.

functools.partial creates a new function with some arguments already filled in. It is useful when you have a general function and want a few project-specific versions of it.

This pattern is useful when setting up processing pipelines. Instead of passing the same configuration through every call, you create a pre-configured function once:

partial is often cleaner than a lambda for argument pre-filling. It keeps the intent clear and works well with functions that have many keyword arguments, which is common in AI libraries and SDKs.

References