A Literature Survey and Experimental Evaluation of the State-of-the-Art in Uplift Modeling: A Stepping Stone Toward the Development of Prescriptive Analytics

Abstract

Prescriptive analytics extends on predictive analytics by allowing to estimate an outcome in function of control variables, allowing as such to establish the required level of control variables for realizing a desired outcome. Uplift modeling is at the heart of prescriptive analytics and aims at estimating the net difference in an outcome resulting from a specific action or treatment that is applied. In this article, a structured and detailed literature survey on uplift modeling is provided by identifying and contrasting various groups of approaches. In addition, evaluation metrics for assessing the performance of uplift models are reviewed. An experimental evaluation on four real-world data sets provides further insight into their use. Uplift random forests are found to be consistently among the best performing techniques in terms of the Qini and Gini measures, although considerable variability in performance across the various data sets of the experiments is observed. In addition, uplift models are frequently observed to be unstable and display a strong variability in terms of performance across different folds in the cross-validation experimental setup. This potentially threatens their actual use for business applications. Moreover, it is found that the available evaluation metrics do not provide an intuitively understandable indication of the actual use and performance of a model. Specifically, existing evaluation metrics do not facilitate a comparison of uplift models and predictive models and evaluate performance either at an arbitrary cutoff or over the full spectrum of potential cutoffs. In conclusion, we highlight the instability of uplift models and the need for an application-oriented approach to assess uplift models as prime topics for further research.

Response transformation and profit decomposition for revenue uplift modeling

“…To capture a causal link between a marketing activity and customer behavior, uplift models require data from two groups, the treatment and the control group (Devriendt et al, 2018). Such data is gathered through randomized trials in previous work; often in the form of A/B tests in e-commerce settings.…”

“…The term was coined in the direct marketing literature where the modeling goal is often to predict how a customer will react to a marketing stimulus (e.g., Baesens et al, 2002). Uplift models also consider a direct marketing setting but estimate the differential change in response behavior due to the marketing activity (Devriendt et al, 2018). This way, an uplift model accounts for the causal link between the action and customer response.…”

“…However, a model that gives biased estimates of the CATE -or foregoes its estimation altogether -can still be a valuable uplift model if the relative order of customers in the ranked list is accurate. Uplift modeling strategies such as the class variable transformation, which display appealing results in several benchmarks (Devriendt et al, 2018;Gubela et al, 2019;Kane et al, 2014), and their extensions as proposed in this paper, belong to the latter category. In view of recent advancements in the literature on treatment effects and the development of several highly recognized methods such as causal forests , causal boosting (Powers et al, 2018), causal BART , or the X-learner of Künzel et al (2019), it is interesting to examine whether these CATE estimators are a suitable vehicle for revenue uplift modeling.…”

A survey and benchmarking study of multitreatment uplift modeling

“…The MTUM literature is still limited. This study categorizes the different MTUM approaches according to the classification proposed by Devriendt et al (2018) for binary uplift models. The authors distinguish two main methods to obtain uplift estimates: the data preprocessing approach and the data processing approach.…”

“…The binary treatment uplift models presented in the survey by Devriendt et al (2018) can be extended to indirectly predict the optimal treatments in the MTUM scenario. The NUA is a data processing method in which uplift is estimated indirectly.…”