Novelty effect
A temporary rise or dip caused by a change being unfamiliar, which fades after people stop exploring it or learn the new behavior.
See it
What it is
A novelty effect is a temporary behavior change caused by the change itself being new. A redesigned navigation item may get a burst of clicks from curious existing users, then settle back once they have inspected it. The reverse has its own name, change aversion or the primacy effect: a useful change can start badly while people unlearn the old path.
Reach for this explanation when a treatment effect is strongest just after first exposure and decays with time. Plot lift by time since each user's first exposure, not only by calendar date, and separate existing users from people whose first experience was already the new one. Run long enough for repeat behavior to appear, especially when the metric is retention or habitual use.
Gotcha: any early spike is not automatically novelty. A launch email, weekday mix, instrumentation change, or a small first cohort can draw the same shape. Looking only at the final average can hide a treatment that helped for three days and harmed afterward, while waiting until the line flattens can become another excuse to stop at a convenient result. Predeclare the time slices and decision window.
Ask AI for it
Test this experiment for a novelty effect. Build a user-level table with assigned_variant, first_exposure_at, outcome, and days_since_first_exposure; bucket days 0 to 2, 3 to 6, 7 to 13, and 14 plus; and report treatment lift with a 95 percent confidence interval in every bucket. Fit the interaction model in Python with statsmodels.formula.api.logit using the formula 'outcome ~ treatment * C(time_bucket)', then plot the interaction estimates separately for existing users and users acquired after launch. Use the predeclared primary window for the ship decision even if an earlier bucket looks better.