AlgoMaster Logo

pytest Framework

High Priority39 min readUpdated September 21, 2026
Listen to this chapter
Unlock Audio

pytest is the third-party framework that most Python projects actually use. You'll see how pytest discovers tests, why plain assert is enough (no assertEqual zoo), and the handful of helpers (pytest.raises, pytest.approx, markers) that make day-to-day test writing pleasant.

Why pytest

unittest, the test framework that ships with the standard library, works, but the API is heavy. Every check goes through a method (assertEqual, assertTrue, assertIn, assertRaises, and a couple dozen more), every test lives inside a TestCase subclass, and setup happens through setUp/tearDown boilerplate. That style was familiar to anyone coming from JUnit in the early 2000s. It's also a lot of typing for what should be simple checks.

pytest takes a different stance. A test is a plain function whose name starts with test_. A check is a plain assert. There's no class to inherit, no method to look up, no special imports for most cases. The framework handles the rest: collecting tests from your files, running them, and producing a readable failure report when something breaks.

The result is shorter tests, less ceremony, and (this matters more than it sounds) failure messages that actually show you what went wrong. Where unittest shows AssertionError: 99.95 != 99.96, pytest shows the full expression, the values of every sub-expression, and the diff. You'll see this in action in a moment.

pytest is not in the standard library. It's a community project, but it's the de facto standard. The big open-source Python projects, the major web frameworks, the data and ML libraries, all use it. If you're writing tests in 2026, this is the framework to learn.

Installing pytest

A single pip install gets you the framework and the pytest command:

Confirm the install:

Versions move forward, so your number may differ. Anything in the 8.x range behaves the same for what's in this chapter. From here on, every pytest command in the chapter assumes the binary is on your PATH. If it isn't, run python -m pytest instead, which calls the same entry point through the interpreter.

A typical project keeps tests in a tests/ folder next to the application code:

The tests/ directory doesn't need an __init__.py in most setups. pytest collects from it directly. Some larger projects do add one to control import paths, but you don't need to start there.

Your First pytest Test

Here's a minimum working example. The application code:

The test:

Run it from the project root:

The dot after the file name is the test result. One dot per test: . for pass, F for fail, E for error during setup, s for skipped, x for expected-fail. The summary line at the bottom totals everything up.

That's the whole loop. Write a function whose name starts with test_, put an assert inside, run pytest. There's nothing else to import, nothing to inherit from.

Test Discovery Rules

pytest finds tests by following a small set of conventions. The rules are simple enough to memorize, and they're how the framework gets away with so little ceremony.

ItemRule
Test filesNamed test_*.py or *_test.py
Test functionsNames start with test_
Test classesNames start with Test, no __init__ method
Test methods (inside Test classes)Names start with test_

Anything not matching these patterns is ignored. A helper file called helpers.py in your tests/ folder won't be collected, even if it contains functions whose names start with test_. A function called check_total inside test_pricing.py won't be collected, because its name doesn't start with test_. A class called OrderTests won't be collected, because its name doesn't start with Test.

The "no __init__ method" rule on test classes catches a few people off guard. pytest can't instantiate a class with constructor arguments, so if you write def __init__(self, ...) on a Test* class, pytest skips it with a warning. Use fixtures for setup instead.

Here's a quick demonstration of all three forms in one file:

Run pytest and all three are picked up:

The :: notation is pytest's way of pointing at a specific test. You can copy any of those identifiers and pass it back to pytest to run just that one:

This shows up everywhere: in CI failure output, in editor integrations, in the --lf "rerun last failed" feature. Knowing how to read the :: path is useful.

The diagram shows the two-stage pipeline: collection walks the directory tree and finds matching files, functions, and classes; execution runs everything collection found. Collection happens up front, before any test runs. If you have a syntax error in test_cart.py, you'll see it as a collection error, not a test failure.

Plain assert and Why It's Better

This is pytest's headline feature. You write a normal Python assert, and when it fails, pytest unpacks the expression and shows you what each side evaluated to. The technical term for this is assertion rewriting: pytest rewrites the bytecode of your test functions at import time so that a failing assert produces a rich diff instead of a bare AssertionError.

Compare. Here's an equivalent test in unittest and pytest:

Both fail (the actual total is 54.97, not 99.99). Look at the failure output side by side.

unittest output:

pytest output:

The pytest version shows the failing line, breaks it into its sub-expressions, and tells you which call produced which value. For a small case the difference is minor. For anything more involved, the difference is enormous.

Here's a more interesting failure. Suppose you have a function that builds an order summary dict:

A test with the wrong expected value:

pytest shows a structured diff:

For lists and strings, pytest shows element-level diffs. For nested structures, it points at which key differs. For long sequences, -v (verbose) gives the full side-by-side diff.

That richness is the whole reason people switch off assertEqual and friends. You don't have to remember which assertion method to use for which type. assert works for everything, and the failure output gives you enough information to fix the test (or find the bug) without rerunning under a debugger.

Expecting Exceptions with pytest.raises

Some code is supposed to raise. A discount above 100% should reject. A negative quantity should reject. Pulling stock for an out-of-stock item should reject. You don't want to write try/except boilerplate for every one of those tests; pytest gives you pytest.raises as a context manager.

The pattern is straightforward:

Inside the with block, the code under test must raise the named exception (or a subclass) for the test to pass. If it raises a different exception, that exception propagates and the test errors out. If it raises nothing at all, the test fails with DID NOT RAISE.

You can also check the exception message using match=, which takes a regex:

match= is a re.search, not a full-string match, so you only need a fragment of the message. Useful for asserting that the error is for the right reason without pinning the test to the exact wording.

If you need to inspect the exception (assert on its attributes, for example), the context manager captures it as the as target:

exc_info.value is the exception instance. exc_info.type is the exception class. exc_info.traceback is the traceback object if you ever need it.

One thing to watch for: only the code that needs to raise should be inside the with block. Setting up data or asserting on side effects should be outside.

What's wrong with this code?

The assert result < 0 will never run; the call before it raises. But more importantly, if the setup of total or percent were to raise something unrelated, the test would still pass (pytest.raises would accept the wrong-reason exception). Keep the block tight:

Fix:

Now the only thing the with block guards is the one call you expect to raise.

Comparing Floats with pytest.approx

Floating point math is approximate, and equality checks on floats are a common source of false failures. The classic example:

This bites tests all the time. Sum a list of prices, apply a discount, compare the result to a hand-calculated total, watch the assertion fail by a millionth of a cent.

pytest's answer is pytest.approx, a helper that compares floats with a tolerance:

This passes. pytest.approx defaults to a relative tolerance of 1e-6 (one part in a million), which is loose enough to absorb the usual IEEE 754 noise but tight enough that real bugs still surface.

You can tune the tolerance when you need to:

abs is good when you want "within a cent" or "within a millisecond" semantics. rel is good when the expected value can vary in magnitude (a $5 cart and a $5,000 cart should both pass a "within 0.1%" check).

approx works on collections, too. Comparing two lists of floats element-by-element is one line:

It also works on dict values:

If you're testing anything that involves arithmetic on floats (totals, taxes, discounts, percentages, scientific computation, ML metrics), make approx your default. Plain == on floats is a footgun.

Skips and Expected Failures

Tests aren't always supposed to run. Some depend on an environment that isn't available (a database, a specific OS, a feature flag). Some are for code that's known to be broken right now, where you want to mark the failure as "expected" so it doesn't break the build, but still flip back to a pass once it's fixed.

pytest gives you three tools for this: pytest.skip, pytest.xfail, and pytest.fail. All three can be called as functions inside a test, and they have decorator equivalents (@pytest.mark.skip, @pytest.mark.skipif, @pytest.mark.xfail) for cases where you can decide before the test runs.

pytest.skip and @pytest.mark.skip

Use skip when a test shouldn't run at all under the current conditions:

The decorator form is cleaner when the condition is static:

@pytest.mark.skipif takes a condition:

In the output, skipped tests show as s in the dot summary, and pytest -rs (or -ra for all non-pass reasons) prints the skip reasons:

@pytest.mark.xfail

Expected failure. The test runs, but its result is inverted: if it fails, the suite still passes (recorded as xfail); if it unexpectedly passes, that shows up as xpass, which you can configure to fail the build (strict=True).

xfail is the right tool when you know about a bug and you've written the test that proves the fix, but the fix isn't merged yet. It documents the failure in the test suite without breaking CI. Once the fix lands, the test starts passing, and you remove the marker.

pytest.fail

Force a failure with a message. Useful inside branches where the test should never reach (a defensive "this shouldn't happen" check):

You'd reach for this in tests with multiple branches or post-hoc inspection logic. Most of the time, a plain assert does the job.

Markers Basics

A marker is a tag you attach to a test (or a class, or a whole module) with @pytest.mark.something. pytest treats certain markers as built-in (skip, skipif, xfail, parametrize), and you can define your own for filtering and grouping.

You've already seen skip, skipif, and xfail. parametrize runs the same test against multiple inputs.

Custom markers let you tag tests by category and then run subsets:

Run only the smoke tests:

Run everything except the slow ones:

Combine markers:

There's one gotcha: by default, pytest warns about unknown markers. To register your custom markers properly, add them to pytest.ini (or the [tool.pytest.ini_options] section in pyproject.toml):

With that in place, pytest --markers lists all registered markers along with their descriptions, and the warnings stop. Skipping this step works, but you'll see noise like:

Register your markers. It's a one-line entry per marker, and it gives the next person on the team a place to look for the conventions you're using.

Useful Command-Line Flags

pytest has more flags than you'll ever memorize, but a small set comes up daily. These are worth committing to muscle memory.

FlagWhat it does
-vVerbose. Show each test name and pass/fail status, instead of dots.
-qQuiet. Even less output than the default. Good for CI.
-xStop on the first failure. Don't continue with the rest of the suite.
--maxfail=NStop after N failures. Like -x but with a budget.
-k EXPRRun only tests whose name matches the expression.
-m EXPRRun only tests whose markers match the expression.
--lf / --last-failedRe-run only the tests that failed last time.
--ff / --failed-firstRun failed tests first, then the rest.
-sDon't capture stdout. Let print() show through to the terminal.
--tb=shortCompact tracebacks. --tb=long for full, --tb=line for one-liners, --tb=no for none.
--collect-onlyShow what would run, don't actually run anything.

-x: Stop on First Failure

When you're debugging a regression, the first failure is usually the most useful. The next ten failures might be downstream effects of the same broken state. -x halts the suite at the first red, which gets you to the relevant traceback immediately.

-k EXPR: Filter by Test Name

Run only tests whose names contain a substring:

This matches every test whose qualified name (file, class, function) contains "discount" (case-insensitive). The expression supports and, or, and not:

That runs every discount test except the ones whose name contains "negative".

--lf and --ff: Re-Run Failed Tests

After a failure, the tightest feedback loop is to fix the code and re-run only the test that broke. --lf does exactly that:

If everything is fixed, --lf reports "no tests ran" and skips the work. If some failures remain, only those run.

--ff is the cousin: run the failed tests first, then everything else. Useful when you want fast feedback on the known-broken tests but also want to make sure you didn't break anything else.

-s: Let print() Through

pytest captures stdout during test runs, so print() calls don't show up unless a test fails. When you're debugging by sprinkling prints, that's frustrating. -s disables the capture:

Use it sparingly. The default capture is there because hundreds of print lines from passing tests would drown the actual report.

--tb=short: Compact Tracebacks

The default traceback is helpful but verbose. For a suite with a hundred failures (think: a refactor that breaks an import), the screens of traceback get unmanageable. --tb=short trims each to just the failure location and one line of context:

--tb=line is even more compact (one line per failure). --tb=no suppresses tracebacks entirely, leaving just the summary. All three are good for skimming a large failure set before diving in.

Project Layout and conftest.py

A typical pytest project keeps tests in their own folder, mirroring the application structure:

conftest.py is a special file pytest knows about. It's where you put fixtures and hooks that should be shared across multiple test files in the same directory (and its subdirectories). You don't import it; pytest finds it automatically.

The name conftest.py is reserved, and you'll see it in every real-world pytest project. If you're poking around an open-source codebase and see a conftest.py, that's where the shared test setup lives.

You don't have to put __init__.py in the tests/ folder. pytest can collect from a flat directory without it. Some projects do add it (often to fix import conflicts when two files in different folders have the same name), but it's not the default.

For a one-file project, you can also just drop your tests at the top level:

Running pytest from that directory works the same. The folder layout is convention, not requirement.

A Worked Example

Here's everything together. A small e-commerce module and a test file that exercises the pieces covered in this chapter.

The application code:

The tests:

Running this with the default flags:

To skip the slow test for a quick check during development:

To run only the discount tests:

To re-run anything that failed last time, stopping on the first one:

That's the daily flow. Most of pytest's power comes from combining these few flags, and you'll use them dozens of times a day once it's in your fingers.

A couple of pieces are conspicuously missing from this example: there's no @pytest.fixture for setting up the cart, and there's no @pytest.mark.parametrize for the four TestApplyDiscount cases that all look similar. Fixtures let you share setup code across tests without duplication. parametrize would turn those four discount tests into one parametrized test with four input rows. monkeypatch is a built-in fixture for replacing dependencies during a test.

Quiz

pytest Framework Quiz

10 quizzes