A canary deployment can show a positive trend, get shipped, and then fade a few weeks later. The issue isn’t always the model, it’s the experiment. If the test isn’t designed properly, you end up shipping on noise.

The previous chapter covered how to get a model into an A/B test: shadow mode, interleaving, and canary deployment.

This chapter focuses on what comes next. How to design and run experiments so the results are reliable and actually worth acting on.