Mastering the Mixture Sequential Probability Ratio Test (mSPRT) for A/B Testing

The Mixture Sequential Probability Ratio Test (mSPRT) is a statistical method designed for anytime-valid A/B testing, allowing experimenters to monitor results without inflating false positives. Building on Wald’s original SPRT, mSPRT adapts the framework to two-sample scenarios and is widely used by companies like Uber and Netflix. Compared to safe tests, it offers optimal performance for granular, sequential data, making it a powerful tool for modern experimentation infrastructures.

Table of Links

5 Mixture sequential probability ratio test

5.1 Sequential Testing

As sophisticated A/B testing infrastructure has proliferated, so too have the opportunities to peek at test results [Joh+17]. As we’ve seen, this leads to the unintended consequence of inflating the false positive rate. To take advantage of their infrastructure, then, big technology companies have begun implementing statistical methods that are valid at any time. This field of statistics ...