To Treat, or Not to Treat: Reducing Volatility in Uplift Modeling Through Weighted Ensembles

Abstract

When conducting direct marketing activities, companies strive to know whom to target with a marketing incentive to maximize the campaign effect. For example, which customer should receive churn prevention incentive to minimize overall churn rate? Uplift modeling is a promising approach to answer such a question. It allows to separate customers who would likely react positively to a treatment from those who would remain neutral or even react negatively. However, while different uplift approaches have been proposed, they usually suffer from high volatility and their performance often depends largely on data set and application context. Thus, it is difficult for practitioners and researchers to apply uplift modeling. To overcome these problems, we propose a weighted ensemble of different uplift modeling approaches to reduce volatility and improve robustness. We evaluate the novel approach against single uplift modeling approaches on multiple data sets and find that the ensemble is indeed more robust.

Bridging the Gap: A Systematic Benchmarking of Uplift Modeling and Heterogeneous Treatment Effects Methods

“…In line with current research (Athey and Imbens 2015;Devriendt, Moldovan, and Verbeke 2018;Rößler, Tilly, and Schoder 2021), our results show that most methods suffer from volatility: that is, they perform well for some data sets but poorly for others. We do, however, find that some methods perform better and are more robust than others.…”

“…Overall, as other researchers have already noted (Devriendt, Moldovan, and Verbeke 2018;Rößler, Tilly, and Schoder 2021), the algorithms' performances vary from study to study. While most researchers agree on the weak performance of transformed outcome and class variable transformation, researchers disagree on the performance of other algorithms, such as tree-based ensembles like the causal forest and the uplift random forest.…”

“…Finally, Table 1 reveals that most benchmarking studies used only a few data sets for the evaluation. We can see that only six studies assessed the performance on more than three data sets (Devriendt, Moldovan, and Verbeke 2018; Gubela et al 2019; Rößler, Tilly, and Schoder 2021; Rzepakowski and Jaroszewicz 2012; Sołtys, Jaroszewicz, and Rzepakowski 2015; Zaniewicz and Jaroszewicz 2017). Yet, the data sets used by Rzepakowski and Jaroszewicz (2012), Sołtys, Jaroszewicz, and Rzepakowski (2015), and Zaniewicz and Jaroszewicz (2017) had not initially been collected for ITE estimation.…”

“…Given the complexity of ITE estimation (Gupta et al 2020), we must integrate methods and synthesize findings from both research streams. For example, most methods from uplift modeling and HTE suffer from high variance (Devriendt, Moldovan, and Verbeke 2018; Rößler, Tilly, and Schoder 2021; Wager and Athey 2018); combining methods and findings from both streams might lead to more robust solutions. Similarly, if the goal is to estimate ITEs in the best possible way or make a new contribution in the context of targeting policies, ignoring the very rich literature on either uplift modeling or HTE is a serious omission for both academics and practitioners.…”

E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling

“…Such solutions often address a single-offer treatment, which need to account for a direct comparison between two alternativesto treat or not to treat? [25].…”