A canary can show a positive trend, get shipped, and fade a few weeks later. The issue is not always the model. Sometimes the experiment was too short, underpowered, biased by peeking, or measuring the wrong outcome.

Shadow mode, interleaving, and canary deployment get a model safely onto live traffic. The A/B test is what tells you whether to actually ship it.

This chapter covers what that takes: designing and analyzing experiments so the result is reliable enough to act on.