Hyperparameters determine how training behaves before the model sees a single example. A learning rate that's off by an order of magnitude can make a sound architecture look broken. Batch size interacts with the optimizer and the learning rate, and the wrong choice can stall convergence in a distributed run. Weak regularization is sneakier: offline metrics can look strong while the model quietly overfits in ways that only surface on production data.

Listing the knobs is the easy part. The real difficulty is deciding which ones are worth searching, how much compute to spend on the search, and when a result is trustworthy rather than noise.

This chapter focuses on search strategy: grid search, random search, Bayesian optimization, early stopping, multi-fidelity tuning, and budget allocation.