Performance optimization in C++ is less about clever tricks and more about discipline. Most slowdowns come from a small handful of root causes: bad data layout, the wrong container, surprise allocations, and code paths nobody actually profiled. This chapter walks through how to find those root causes, what the compiler already does for you, and which optimizations are usually worth the time.

Donald Knuth's famous line is, "Premature optimization is the root of all evil." The full quote is more careful: he is warning against optimizing the 97% of code that does not matter, while still encouraging hard work on the critical 3%. The point is not to skip optimization. Find the 3% before touching it.

In practice this means three rules:

Most C++ code does not need to be fast in any micro sense. It needs to be fast in the macro sense, which usually means picking the right data structures, avoiding accidental copies, and not allocating in a hot loop. The compiler will handle most of the rest.

The big mistake is reasoning about performance without measuring. Modern CPUs, caches, branch predictors, and out-of-order execution make intuition unreliable. A loop that "looks slow" might be free because it fits in L1 cache. A loop that "looks fast" might stall on cache misses every iteration. The only reliable answer is measurement.

The diagram is opinionated on purpose. Without a "yes" to "profiled?", every change is a guess. Guesses sometimes help, but they also add complexity that no one can justify later.

There are two kinds of measurement, and they answer different questions.

Profilers answer "where is time being spent across this whole program?" You attach a profiler to a running binary, do realistic work, and the profiler tells you which functions burned the most cycles. Common tools:

Benchmarks answer "how fast is this one piece of code in isolation?" You run a specific function many times under controlled conditions and measure how long each run takes. The standard tool in C++ is Google Benchmark, which handles warm-up, iteration count, and timer noise for you.

A small do-it-yourself benchmark using <chrono>, useful for a quick number without pulling in a library:

That number means nothing on its own. It only matters compared to another version of the same code or a known target ("must process a million prices in under 5 ms"). Always benchmark against a baseline.

std::chrono::high_resolution_clock::now() itself takes tens of nanoseconds. Timing something that runs in 5 nanoseconds means clock noise is bigger than the work. Wrap many iterations inside one timing block.

A few rules for trustworthy numbers:

Reading a Flame Graph

Flame graphs visualize profiler output. The x-axis is total time, not chronological time. Each box is a function. The width of a box is how much time was spent in that function (and its callees). Boxes stacked on top mean "this function called that function".

When reading one, look for wide, flat plateaus. A wide box at the top of the stack is a hot leaf function, which means actual work is happening there. A wide box near the bottom is a hot call site, which means you might be calling something hot too often. Narrow tall stacks are usually fine; they say "we go deep, but we do not stay long".

The first time reading a flame graph, look for surprises: is std::string::operator== 30% of runtime? Is 15% spent in malloc? Those are real findings. Anything below 1% is usually noise.

The compiler does more than many developers realize. Picking the right optimization level often gives a 5x to 50x speedup on the same source code.

-O3 is not automatically faster than -O2. Aggressive inlining can hurt instruction cache locality, and vectorization can pessimize code that does not vectorize cleanly. Measure both.

Two more flags worth knowing:

Link-Time Optimization (`-flto`) lets the compiler optimize across translation units. Without LTO, a function defined in cart.cpp cannot be inlined into order.cpp, even at -O3. With LTO, the linker re-runs optimization with the whole program visible. Builds get slower, binaries get faster.

Profile-Guided Optimization (PGO) is a two-step process: compile an instrumented binary, run it on realistic input, then recompile using the recorded profile. The compiler uses the profile to make smarter inlining and branch-prediction decisions. PGO can give 5-20% on top of -O2 for branchy code like compilers, browsers, and interpreters.

-O0 builds can run 10-100x slower than -O2. Never benchmark a debug build and conclude that C++ is slow.

The biggest wins usually live here, and intuition is often most wrong. Modern CPUs are not bottlenecked on arithmetic. They are bottlenecked on memory.

The numbers above are approximate, but the ratio is what matters. Reading from main memory is roughly 100x slower than reading from L1. A loop that misses cache on every iteration can run 50x slower than the same loop with sequential access, even though the source code is identical.

Two patterns dominate:

Most "this code is mysteriously slow" problems trace back to violating one of these.

Vector vs List

std::vector stores its elements in one contiguous block of memory. std::list stores each element as its own heap-allocated node with pointers between them. The asymptotic complexity tables make std::list look great (O(1) insert anywhere, O(1) splice), but on modern hardware those big-O guarantees are misleading.

The vector is roughly 20x faster, doing identical work. Every list node is a separate heap allocation, scattered across memory. Each step has to follow a pointer, which is a cache miss the prefetcher cannot predict. The vector iterates through one contiguous block and the hardware prefetcher handles it well.

std::list requires roughly 3 pointers of overhead per element (prev, next, allocation metadata). For small POD types like int, that is 3-6x more memory than std::vector, on top of being cache-hostile. Use std::vector by default and use std::list only when O(1) splice or stable iterators under arbitrary insertion are required.

Array of Structs vs Struct of Arrays

A collection of objects can be laid out two ways. The default in C++ is Array of Structs (AoS): each object has all its fields together, and you keep an array of objects. The alternative is Struct of Arrays (SoA): keep one array per field, and an object is the tuple of values at the same index across all arrays.

Why does this matter? Computing the average price across a million products with the AoS layout reads id and stock into cache on every iteration even though they're unused. The SoA layout reads only the prices array, packing 8 doubles into each 64-byte cache line.

The SoA version is roughly 4x faster because it does not waste cache reading fields it ignores. SoA also vectorizes more cleanly, so -O3 can use SIMD instructions on it.

SoA is not free. It makes "give me one product" awkward (you have to gather fields from multiple arrays), and it complicates code. Use it for bulk processing of one or two fields across many objects. Stick with AoS for everything else.

Hot and Cold Field Splitting

A related trick: if a struct has a few fields touched in every hot loop and many fields touched only on the slow path, split them. Put the hot fields in one struct, the cold ones in a separate struct (or behind a pointer), and each cache line carries more hot data.

The hot loop now touches 24 bytes per product instead of ~200. About 2.5 hot products fit in a cache line now, versus a third of a product before.

Algorithmic complexity is necessary but not sufficient. A good default rule for C++ today is "start with std::vector, switch only when measurement forces you to."

A few specifics:

`reserve()` before bulk inserts. std::vector grows by doubling, which means inserting N elements without reserving costs roughly N + N/2 + N/4 + ... ≈ 2N total copies. Reserving first avoids all the intermediate reallocations.

vector::push_back is amortized O(1), but the reallocations along the way move every existing element. For small N this is invisible. For millions of elements, reserve() typically cuts time by 2-3x.

`unordered_map` vs `map`. For lookup only, unordered_map is usually 2-3x faster on average. For sorted iteration or range queries, map is required. For tiny sizes (under ~30 keys), a sorted std::vector<std::pair<K, V>> with std::lower_bound often beats both because of cache locality.

Avoid `std::list` for small POD. As shown above, the cache penalty almost always swamps the algorithmic benefit. Fast splicing is the one case where std::list is the better choice.

Move semantics lets you transfer ownership of resources without copying them. A std::vector<std::string> with a million entries can be moved in a few pointer swaps; copying it allocates a million strings on the heap.

Move is essentially free here because the new vector steals the old one's pointer, size, and capacity. A few things to know:

std::string and some other types use Small String Optimization (SSO) or Small Buffer Optimization (SBO) to avoid heap allocation for short strings. They keep a small fixed buffer inside the object itself (typically 15-23 characters, depending on the standard library), and only fall back to heap allocation when the string outgrows it.

The practical takeaway: storing short identifiers, customer names, and short labels in std::string is usually allocation-free. There's no need to roll your own short-string class. But concatenating strings in a loop can flip a short string into a long one and trigger a heap allocation; reserve capacity up front when the final size is known.

Heap allocations cost roughly 50-200 nanoseconds on a typical glibc allocator, depending on contention. A loop that does ten million allocations spends most of its time inside malloc, not in the user code.

The right convention depends on the size and cost of the type.

A common mistake: passing std::string by value to a getter or comparison function. Each call allocates and copies. Use const std::string& or std::string_view (C++17). std::string_view is a non-owning view, so it has none of the SSO/heap concerns of std::string and is the right type for "read these characters".

The inline keyword started life as a performance hint and has since drifted toward an ODR (one-definition rule) marker. The compiler inlines based on its own heuristics: function size, call frequency, whether the body is visible at the call site, and so on.

What inline does today:

Inlining matters because a function call has overhead (push arguments, jump, return), and inlining lets the compiler optimize across the call boundary. For tiny accessor functions like getPrice(), inlining is almost always a win. For a 200-line function, it inflates code size and can hurt instruction cache.

LTO (link-time optimization) helps here. Without LTO, the compiler cannot inline a function defined in another .cpp file. With LTO, the linker sees all source and can inline freely. If the hot path crosses translation units, turning on -flto often gives a meaningful gain.

CPUs guess which way a branch (if, loop condition, switch) will go and start executing the predicted path. If they guess right, work continues. If they guess wrong, the pipeline flushes and restarts, costing 10-20 cycles per mispredict.

The classic illustration: summing a random array versus a sorted one.

Same data, same result, 3x faster after sorting. The sorted version has predictable branches (the if is false for the first half of the array, then true for the rest), so the CPU's branch predictor is right most of the time.

C++20 added [[likely]] and [[unlikely]] attributes to give the compiler hints about which branch to favor in code layout:

Use these sparingly. The compiler is usually right without hints. Use them only when profiling shows a branch mispredict problem and the hot side is known.

new, malloc, and any container that grows (vector reallocation, unordered_map rehash, string append) call into the allocator, which is not free.

Patterns that reduce allocation pressure:

Arena allocators (also called bump allocators) hand out memory by advancing a pointer through a pre-allocated chunk. Freeing is either nothing (free the whole arena at once) or trivial (decrement the pointer). Pool allocators preallocate a fixed-size chunk and recycle slots. They can cut allocation cost from "hundreds of nanoseconds" to "a few cycles" in the right scenario.

A new for a 100-byte object typically costs 50-100 ns on glibc. An arena allocation for the same object can cost 1-2 ns. In a tight loop, that is a 50x difference in allocation overhead alone.

A few patterns show up often in code that has been "optimized" without measurement:

The thread running through all of these is the same: optimization without measurement is superstition. The principle that holds up best is C++'s own design rule, often phrased as "don't pay for what you don't use, and what you do use should be as efficient as a hand-written version." Trust that principle, and let the profiler point at the actual cost.