1. Targeting for long-term outcomes
Jeremy Yang, Dean Eckles, Paramveer Dhillon, Sinan Aral
Decision-makers often want to target interventions (e.g., marketing campaigns) so as to maximize an outcome that is observed only in the long-term. This typically requires delaying decisions until the outcome is observed or relying on simple short-term proxies for the long-term outcome. Here we build on the statistical surrogacy and off-policy learning literature to impute the missing long-term outcomes and then approximate the optimal targeting policy on the imputed outcomes via a doubly-robust approach. We apply our approach in large-scale proactive churn management experiments at The Boston Globe by targeting optimal discounts to its digital subscribers to maximize their long-term revenue. We first show that conditions for validity of average treatment effect estimation with imputed outcomes are also sufficient for valid policy evaluation and optimization; furthermore, these conditions can be somewhat relaxed for policy optimization. We then validate this approach empirically by comparing it with a policy learned on the ground truth long-term outcomes and show that they are statistically indistinguishable. Our approach also outperforms a policy learned on short-term proxies for the long-term outcome. In a second field experiment, we implement the optimal targeting policy with additional randomized exploration, which allows us to update the optimal policy for each new cohort of customers to account for potential non-stationarity. Over three years, our approach had a net-positive revenue impact in the range of $4-5 million compared to The Boston Globe’s current policies.
🎯How can we learn to target interventions when we care about outcomes that are only observed after a delay?🗓️
— Dean Eckles (@deaneckles) November 2, 2020
This comes up in settings as varied as clinical trials (5 yr all-cause mortality) and marketing (customer lifetime value).
Our new paper:https://t.co/0TZNjc1phT pic.twitter.com/Pa4nVfUg8N