A/B and multivariate testing
A/B testing is a quantitative user research method that detects how different UI designs may change your product's performance. Designers create two versions of the same interface and show them to different groups of users to see which version performs best.[1] Commonly, only a single variable is different, for example, a CTA or a navigational bar.
When you decide to test several design elements such as typefaces, button placement, and alternative icons at once, it's called multivariate testing.
What makes A/B testing effective?
- Choose a variable(s) to test. It can be a CTA, hero image, or other UI elements that can drive significant improvements.
- Define a goal. Choose one metric to analyze and create your hypothesis about possible outcomes.
- Split your audience equally and randomly. Avoid splitting page traffic between different groups by gender or age — you won't get the insights you're looking for.[2]
A/B testing is cost-effective, simple to implement, and perfect for resolving any differences of opinion among team members.
Pro Tip: Make sure to run the tests long enough to produce useful data. It’ll prevent your team from making rash decisions based on small numbers.
References
- Quantitative User-Research Methodologies: An Overview | Nielsen Norman Group