Control group
The randomly assigned group that keeps the old experience, giving you a fair baseline for what would have happened without the change.
See it
What it is
The control group gets the existing experience while the treatment group gets the change. Assign people randomly and at the same time, then the difference between their outcomes estimates what the change caused. Without that concurrent comparison, a payday, a holiday, or a new ad campaign can take credit for the lift. The Medical Research Council's 1948 streptomycin trial is the usual origin story for random allocation, and product experiments borrow the machinery unchanged.
Reach for a control whenever you are testing a product change, price, message, or workflow. Choose the unit before launch: users for a personal experience, accounts when teammates affect one another, or geographies when ads spill across people. Keep assignment stable so the same unit does not bounce between the old and new experiences.
Gotcha: 'everyone before launch' is not a control group; it is a historical baseline exposed to different conditions. Also check for a sample ratio mismatch, where the observed split is far from the planned split, before reading any outcome. Bots, eligibility rules, broken flag evaluation, and logging only one variant can all make a randomized test look convincing after its comparison has already failed.
Ask AI for it
Set up this A/B test in PostHog with a feature flag whose variants are control and treatment, assigned at the user level with a 50/50 split. Keep assignment sticky by distinct_id, log one experiment_exposure event only after the flag is evaluated, and analyze everyone by their assigned variant using intention-to-treat. Before comparing conversion rates, run a chi-square sample ratio mismatch check against the planned split and stop the analysis if p is below 0.01. Report group counts, exposure counts, conversions, absolute lift, relative lift, and a 95 percent confidence interval.