A/B Testing Infrastructure

An A/B test randomly assigns users to two or more variants of a product experience, measures how each variant performs on a chosen metric, and uses the difference to decide which version to ship. The infrastructure that powers it is what lets a company go from "we think the new checkout flow is better" to "the new checkout flow lifted revenue per visitor by 1.8% with 95% confidence over four weeks on 12 million users."

A/B testing is a system design problem disguised as a product practice. Biased random assignment, statistically meaningless metrics, network effects contaminating the control group, pipelines that aggregate events incorrectly: each can turn a careful experiment into a confident wrong answer.