Defining control groups

Control groups are sets of customers who keep using your current solution while others test the new version. For example, when testing a new onboarding email sequence, some customers receive existing emails (control group) while others get the new ones (test group). These control group customers provide the baseline for measuring if the new version works better.

Creating balanced control groups means choosing customers with similar characteristics for both groups. If your test group includes both enterprise and small business customers, your control group needs the same mix. The only difference between the groups should be the change you're testing. While random assignment often creates this balance naturally, always check if your groups truly match in size, type, and behavior patterns.

External factors can affect how your customer groups behave during the test. Seasonal changes, marketing campaigns, or product updates might influence one group differently than another. Also ensure each group has enough customers — small groups might show random variations that look like real differences. Both control and test groups should be large enough to represent your typical customer base.^[1]