Research: When A/B Testing Doesn’t Tell You the Whole Story

How marketers can target high risk customers by identifying the characteristics that correlate with high sensitivity to a given intervention.

Eva Ascarza

Jun 23, 2021 • 6 min read

5.00 (2)

Read Article Review Share

Marketing

A/B Testing

Customer Retention

Loyalty Marketing

Market Segmentation

Summary

Eva Ascarza shares new research based on field experiments with over 14,000 customers that suggests that the traditional A/B testing churn prevention strategy used by marketers isn't as effective in reducing churn rates and maximizing ROI. The author argues that instead, firms should use A/B test data alongside customers’ behavioral and demographic data to determine which subgroup of customers will be most sensitive to the specific intervention that’s being considered. As such, they can easily target their campaigns at the most receptive customers to mitigate churn and boost ROI.

Takeaways

A better strategy for reducing churn rates and increasing ROI is by identifying the characteristics that actually correlate with high sensitivity to a given intervention so that marketers can proactively target their campaigns at the customers who will be most receptive to them.
The traditional churn prevention method used by marketers where they identify which customers are most likely to churn, and then run A/B tests to determine whether a proposed retention intervention will be effective at retaining those high-risk customers, isn’t always the best way to maximize ROI on marketing spend.