C++ gives a lot of rope, and most of it ends up tangled around pointer arithmetic, uninitialized memory, and threads racing for the same byte. This chapter covers the tools that turn cryptic crashes and silent corruption into real, fixable bugs: a disciplined debugging mindset, gdb (with an lldb mapping for Mac users), Valgrind, and the family of sanitizers that ship with g++ and clang++.

Tools matter, but the approach matters more. Most slow debugging sessions come from skipping steps and guessing. A reliable loop:

C++ makes step 1 painful because undefined behavior can hide for a long time. A use-after-free might look fine for a thousand runs and then crash in production at 3 AM. The tools below exist to drag that hidden behavior into the light.

The diagram is a starting point, not a flowchart to follow blindly. Many bugs need two tools (Valgrind to find the leak, then gdb to inspect the allocation site). The key is matching the symptom to the right tool fast, instead of staring at the code hoping the bug reveals itself.

Debuggers can only show what the binary preserved. An optimized release build strips most of that out. Recompile with debug-friendly flags before starting a session:

What each flag does:

-O0 typically makes code 3-10x slower. For long-running tests, -Og keeps most debug info while costing only 1.2-1.5x. Switch to -Og when the -O0 test takes too long.

A common mistake is debugging a release binary, hitting a line in gdb, and seeing <value optimized out> for every variable. The compiler put that variable in a register that got reused, and the debug info can't say where it lives now. When that happens, rebuild with -O0 or -Og.

gdb is the GNU debugger. It runs a program, pauses at any line, inspects variables, walks the call stack, and steps through code one statement at a time. On Linux it's the default. On macOS, Apple ships lldb instead, which is similar enough that the mapping table near the end of this section covers the differences.

Launching and Attaching

Two ways to start a session:

Attaching fits when a server has been running for hours and needs inspection without being killed. The kernel pauses the process while gdb is attached, so don't attach to a production server with latency-sensitive traffic.

A core dump (covered later) loads with gdb ./cart core.

Core Commands

The commands used ninety percent of the time inside gdb:

gdb remembers the last command, so pressing Enter on a blank line repeats it. That helps with next for stepping through a loop body line by line.

Conditional Breakpoints

A plain breakpoint fires every time. Inside a loop that runs a million times, that's useless. Conditional breakpoints fire only when a condition is true:

Now the program runs at full speed until it hits line 42 with quantity > 100. Conditions can also be added to existing breakpoints with condition <bp-number> <expr>.

Inspecting STL Containers

Modern gdb (version 7.0 and later) ships with Python pretty-printers for the standard library. With them enabled, p cart.items prints a readable vector instead of raw internal pointers:

Raw pointers and _M_start, _M_finish mean the pretty-printers aren't loaded. On most Linux distros, sourcing the libstdc++ pretty-printers in ~/.gdbinit fixes it. The libstdc++ source tree includes a python directory with the printers; the standard one-line setup in ~/.gdbinit enables them.

A Real gdb Session

A buggy Cart program that demonstrates how the commands fit together:

The bug is i <= items.size() instead of i < items.size(), which reads one past the end. A walk-through under gdb:

The garbage value for items[2] is the smoking gun. i == 2 is past the end of a two-element vector, so items[i].price reads whatever memory happens to be there. The fix is to change <= to <.

On macOS, the system debugger is lldb, not gdb. The model is the same; the commands differ slightly. A rough mapping:

The short forms (b, r, n, s, c, p, bt) work in both. Switching between Linux and Mac often, sticking to the short forms saves muscle memory.

gdb works well when the bug location is roughly known. Valgrind is the right tool when memory is being corrupted somewhere with no obvious location. Memcheck, Valgrind's flagship tool, runs a program in a virtual CPU and instruments every memory access. It catches leaks, invalid reads and writes, use-after-free, and uninitialized memory reads, all without recompilation.

Install and Run

On Debian-based Linux:

On Fedora or RHEL, the package is also called valgrind. On macOS, official Valgrind support stops at older OS versions, which is one reason ASan is more popular on Mac. Run it on Linux or inside a Linux container instead.

The --leak-check=full flag asks Valgrind to list each leak with a stack trace showing where the memory was allocated, not just a summary.

Reading the Output

Take this leaking cart:

Compiled with g++ -std=c++17 -g -O0 leak.cpp -o leak and run under Valgrind:

The four leak categories matter:

Valgrind also catches invalid reads/writes, which are the symptom of buffer overruns and use-after-free:

"0 bytes after a block of size 400" means a read one element past the end of a 100-int array. That's the same off-by-one bug as the gdb example, but Valgrind finds it without manual breakpoints.

Valgrind runs programs 10-50x slower than native, sometimes more for memory-heavy workloads. Use it on unit tests and small repros, not on full integration test runs.

When Valgrind Misses Things

Memcheck doesn't catch:

It's still the most reliable tool for finding leaks and heap corruption in code that can't be recompiled with sanitizers.

Sanitizers are compile-time instrumentation built into g++ and clang++. Add a -fsanitize=... flag, recompile, and the resulting binary checks itself at runtime. They're faster than Valgrind, catch more categories of bugs, and integrate with gdb for live debugging when something fires.

The five sanitizers worth knowing:

ASan and UBSan typically combine, since they don't conflict and catch different things:

-O1 is a sweet spot: the compiler keeps stack frames and variables readable, but inlining helps the sanitizer produce cleaner stack traces. Pure -O0 works too, just slower.

The loop is short on purpose. Sanitizers turn the slow "wait for a customer to hit the bug" debugging cycle into "compile, run a test, read the trace, fix" in under a minute.

ASan: Use-After-Free Example

An Order built on a heap-allocated buffer that gets freed too early:

createOrder returns a pointer to a stack-allocated Order that's already destroyed by the time main reads description. Built with ASan:

ASan reports exactly which variable (o), which function it came from (createOrder), and which line read it (order.cpp:19). No guessing.

UBSan: Signed Overflow Example

Signed integer overflow in C++ is undefined behavior. UBSan catches it:

The fix depends on context: use int64_t, check inputs, or use __builtin_add_overflow if you need to detect overflow at runtime.

TSan: Data Race Example

A Cart::addItem called from two threads without a mutex:

TSan names both threads, the address, and the source lines that touched it. Fixing this means adding a std::mutex around items.push_back, or using a thread-safe queue.

ASan slows programs about 2x and roughly doubles memory use. UBSan adds maybe 20%. TSan slows programs 5-15x and uses 5-10x more memory. Don't ship binaries with sanitizers on.

Sanitizers vs Valgrind

When to choose which? The honest comparison:

The short rule: default to ASan + UBSan in CI, use Valgrind when recompilation isn't possible (third-party binary, plugin, or a bug that only repros in an uncontrolled release build).

Which Sanitizer to Use When

When a program crashes in production, attaching a debugger is often not an option. A core dump is a snapshot of the process memory at the moment it died. It loads later in gdb for state inspection.

Core dumps are usually disabled by default. Enable them in the shell:

This sets the maximum core size to unlimited for the current shell. To make it persistent, add it to /etc/security/limits.conf. On many distros, core files are routed to systemd-coredump and viewed with coredumpctl:

When cores land in the working directory (older setups), load one with:

bt full prints the backtrace with local variables for each frame. That's usually enough to figure out which pointer was null, which vector was out of bounds, or which assertion fired.

The catch: the core file is only useful when the binary was built with -g and the exact same binary exists on the analysis machine. Production binaries are often stripped, in which case keep the unstripped version in a build artifact archive.

Debuggers are interactive; logs are passive. Each has its place:

Good logs look more like structured records than free-form text:

Key-value pairs make logs grep-friendly and machine-parseable. The opposite, prose-style logging like "Customer purchased some stuff for a lot of money", is useless six months later when diagnosing a bug. Real systems usually use a library (spdlog, fmtlib, or one wired to the company's log pipeline), but the principle is the same: structured first, readable second.

A common pattern: keep logs cheap in production (INFO and above), but flip a runtime flag to DEBUG when investigating. Don't litter the code with std::cout calls and then comment them out. Use a real log level.

Dynamic tools (the debugger, Valgrind, sanitizers) all need the program to run. Static analyzers read the source without running it and flag suspect patterns: dead code, possible null dereferences, missing override, unused variables, suspicious casts.

Two worth knowing:

A quick sanity run:

Static analysis catches a category of bugs neither debugger nor sanitizer ever sees: code that never gets executed because of branch coverage gaps. Treat it as a pre-flight check, not a substitute for runtime tools.

Modern IDEs wrap gdb or lldb with a visual interface. VS Code with the CodeLLDB extension (or the C/C++ extension's built-in gdb integration) supports breakpoints by clicking the gutter, hover-to-inspect variables, and a watch panel for expressions. CLion ships its own bundled debugger UI. The commands above (break, step, next, print) all map to UI actions, so the muscle memory transfers either way. For day-to-day work, a GUI is often faster; for remote servers, scripted sessions, or core dump triage, command-line gdb is still the right tool.