A/B ⚖ Testing with Multiple Metrics

Angelina Yang
4 min readDec 1, 2022

A lot of literature references and guidance about A/B testing anchor on tests based on one single comparison or one single metric. Some of my friends working in tech also shared that they typically focus on one primary metric when performing experiment designs (e.g., study design and sample size calculation etc.).

I’m curious if this is a common practice in the tech industry (especially the non-biotech industries).

Do you factor in multiple metrics when performing experiment design at your company?

--

--