AlgoMaster Logo

Working with JSON

Medium Priority24 min readUpdated September 21, 2026
Listen to this chapter
Unlock Audio

JSON is how most web services, configuration files, and inter-service messages move data around. Python's standard library ships a json module that converts between JSON text and Python objects with two pairs of functions: loads/load for parsing and dumps/dump for writing. This lesson covers the type mapping, the four core functions, the formatting options that matter, the errors you'll hit, and how to extend the encoder and decoder for types JSON doesn't know about.

Why JSON Shows Up Everywhere

When an online store fetches a product catalog from a backend, the response is almost always JSON. When a mobile app posts a new order, the body is JSON. When a config file ships with a service, it's probably JSON. The format is human-readable, language-neutral, and maps cleanly onto the data structures every language already has: lists, dicts/maps, strings, numbers, booleans, and null. Python's json module is the official translator between JSON text and Python objects.

The JSON true became Python True, the JSON object became a dict, and the price is a real float you can do arithmetic on. That's the whole appeal: parse once, work in Python.

JSON Types vs Python Types

JSON has six value types. Python has more, but the json module only knows how to translate between this fixed set. Anything outside the table needs custom handling, which we'll get to later.

JSONPythonNotes
objectdictKeys are always strings in JSON.
arraylistAlways a list, never a tuple, on decode.
stringstrUnicode by default.
number (int)intNo size limit on the Python side.
number (float)floatIEEE 754 double precision.
true / falseTrue / FalseLowercase in JSON, capitalized in Python.
nullNoneOne direction is obvious, the other catches people.

Going the other way, Python tuple becomes a JSON array, so a round trip turns tuples into lists. Sets, datetime objects, Decimal, and your own classes are not in the table at all and will raise TypeError unless you teach the encoder how to handle them.

Notice the asymmetry: tuples encode as arrays, but arrays always decode as lists. If you need a tuple back, convert after decoding.

Parsing JSON: loads and load

There are two parsing functions, and the only difference is where the JSON comes from. Use loads (load string) when you already have the JSON as a str. Use load (load file) when you have a file object opened in text mode.

For files, open the file and pass the file object directly. The function reads and parses in one step.

The encoding="utf-8" matters. JSON is officially UTF-8 by default, and product names, customer names, or address fields may contain characters outside ASCII. Opening without an explicit encoding falls back to the platform default, which on Windows can be cp1252 and silently mangle non-Latin text.

A file containing a JSON array of records (a catalog of products, a list of orders) reads exactly the same way; the whole file is still one JSON value. json.load returns the list and you iterate over it normally.

You can read with pathlib too, which lets you skip the open boilerplate when you just need the text:

We'll stick with open for the rest of the examples.

Writing JSON: dumps and dump

The output side mirrors the input side. dumps returns a JSON string. dump writes JSON directly to an open file.

Two things to notice. First, JSON booleans are lowercase (false, not False). Second, the output is one long line with no whitespace beyond the separators. That's the most compact form, which is what you want when shipping data over a network. It's also unreadable for humans.

To write to a file:

The file ends up with the same single-line output you saw from dumps. The choice between dump and dumps is just "do I have a file or a string?". dump is slightly more efficient because it doesn't build the full string in memory before writing, though for normal payload sizes you won't notice.

Pretty Printing: indent, sort_keys, separators

The default output is compact. For anything a human will read, like a config file, a sample fixture, or a snapshot in a test, pass indent to get a readable layout.

indent=2 means two spaces per level. indent=4 is also common. You can pass any non-negative integer, or even a string (indent="\t" for tabs).

By default, the output preserves dict insertion order. That's fine until you want a stable diff between two versions of the same file. sort_keys=True orders keys alphabetically at every level, which makes JSON output deterministic regardless of how the dict was built.

The third formatting knob is separators. By default, it's (", ", ": "): comma-space between items, colon-space between key and value. For the most compact output (no spaces at all), pass (",", ":").

That extra trimming matters when you're sending millions of small records over a network or storing them in a column that has a size limit. It also produces canonical output that's stable byte-for-byte across writes.

A practical pattern: wrap the two modes in helpers and call the one that fits the job.

ensure_ascii and Non-ASCII Text

By default, json.dumps escapes any non-ASCII character as a Unicode escape sequence. That keeps the output strictly 7-bit ASCII, which used to matter for systems that couldn't handle UTF-8 cleanly.

The é becomes é by default. Technically correct (the JSON parser will decode it back to é), but ugly, larger, and unhelpful when you're reading the file. Almost every modern system handles UTF-8, so ensure_ascii=False is usually what you want.

When writing to a file, pair this with encoding="utf-8" on the open call:

The file ends up containing the actual characters, encoded as UTF-8 bytes. Readers using the same encoding (which is the default for most tools today) see the original text.

Handling Errors: JSONDecodeError

Parsing user-supplied or network-fetched JSON fails sometimes. The data might be truncated, malformed, or just not JSON. The json module raises json.JSONDecodeError, a subclass of ValueError, with helpful attributes for diagnostics.

JSONDecodeError carries the message, the line and column where parsing failed, and the absolute character position. That's enough to point a user at the broken spot in their file. The error message itself (Expecting ',' delimiter) is the parser's best guess at what it wanted to see next.

A typical catch in production code distinguishes between "the file isn't there" (FileNotFoundError), "the file has bad JSON" (json.JSONDecodeError), and "the JSON is valid but missing fields you need" (KeyError). Each one deserves a different response.

A subtler failure: the JSON parses fine, but the structure isn't what you expected. The parser doesn't know your data shape; it just turns text into Python objects. Validating that the result has the keys you need is your code's job. Libraries like pydantic and jsonschema exist for this; we'll skip them and stick to plain checks.

Types JSON Can't Handle by Default

JSON only knows about the seven types in the mapping table. If you try to encode anything else, the encoder raises TypeError.

What's wrong with this code?

A set has no native JSON form. JSON arrays are ordered; sets are not. The encoder doesn't guess, it errors.

Fix: convert the set to a list before encoding. Sorting gives stable output too.

The same problem applies to datetime, Decimal (which is common in money calculations because float can't represent 0.1 exactly), and any custom class.

The json module isn't being lazy. JSON is a strict format with a closed set of types, and the standard library refuses to invent a convention for everything else. That's where custom encoders come in.

Custom Encoders: The default Function

The simplest extension point is the default argument to json.dumps and json.dump. When the encoder hits an object it doesn't know how to serialize, it calls default(obj) and uses whatever JSON-serializable value you return.

default is called once per unknown object. It must either return a JSON-serializable value or raise TypeError. Re-raising is important: if you silently return None, you'll produce JSON that looks fine but has missing data, which is harder to debug than a clean failure.

Decimal becomes a string here, not a float. That preserves precision: Decimal("29.99") is an exact value, but float(Decimal("29.99")) would round it to the nearest IEEE 754 double. For money fields, strings are the safe choice. The downside is that the receiving end has to know to parse those strings back as Decimal.

Custom Encoders: Subclassing JSONEncoder

For more structured handling, subclass json.JSONEncoder and override default. The behavior is the same, but the encoder is reusable, configurable, and easy to plug into multiple call sites.

The cls= argument tells json.dumps to use your subclass. Calling super().default(obj) for anything you don't recognize delegates back to the parent, which raises the standard TypeError. That's better than raising your own, because users can catch TypeError and get the same message they'd see from plain json.dumps.

For application-defined classes, the convention is to define a to_dict method on the class and have the encoder call it. That keeps each class responsible for its own JSON shape, and the encoder stays a thin dispatcher that says "if you can describe yourself, do so". dataclasses.asdict offers a terser way to do this for data classes. For now, the explicit to_dict pattern is fine and works on any class.

Custom Decoders: object_hook

Parsing has its own extension point. The object_hook parameter is a function that gets called for every JSON object (Python dict) the parser produces. Whatever it returns replaces the dict in the result.

The hook runs bottom-up: inner objects are passed before outer ones. That means by the time object_hook sees the outer order, the inner items (if any) have already been transformed. Useful for nested structures.

A common application is turning a JSON object back into a domain class. If the JSON has a "type tag" field that says which class it is, the hook can look at it and dispatch:

The _type field is a convention you choose, not a JSON feature. The encoder side has to include it when writing, and the decoder side knows to look for it. If you don't control both sides, the type tag won't be there and you'll fall back to plain dicts.

For projects that need decimal precision back on the way in, there's parse_float:

parse_float is called for every JSON number that has a decimal point or exponent. There's also parse_int for integers and parse_constant for Infinity, -Infinity, and NaN.

JSON Lines: One Record Per Line

Standard JSON forces you to load the whole file before you can use any of it. For logs, exports, or streaming data, that's wasteful. The JSON Lines format (sometimes called NDJSON, for newline-delimited) puts one JSON value on each line, with no enclosing array. Each line is independently parseable.

A orders.jsonl file might look like:

You read it line by line and parse each line on its own:

Writing follows the same pattern: open a file, loop through your records, and write each one as json.dumps(record) + "\n".

When to use JSON Lines: log records, append-only data, exports from databases, anything where records arrive over time. When to stick with regular JSON: small configuration files, structured objects that need to be one whole tree, anything a human will hand-edit.

Putting It Together: Round-Tripping a Wishlist

Here's a compact example that ties the pieces together: a Product class with a to_dict method, a custom encoder, and a decoder hook that rebuilds the class from the JSON.

The _type tag is the bridge between JSON shape and Python class. The encoder stamps it on the way out; the decoder reads it on the way in. Decimal stays exact because we encode it as a string. The diagram below traces a single record through the full pipeline.

The encoder flattens Python-only types into JSON-friendly ones, and the decoder rebuilds them on the way back. The JSON text in the middle is the source of truth that any other tool, in any language, can read.

Quiz

Working with JSON Quiz

10 quizzes