The foundations of cost-sensitive causal classification

Abstract

Classification is a well-studied machine learning task which concerns the assignment of instances to a set of outcomes. Classification models support the optimization of managerial decision-making across a variety of operational business processes. For instance, customer churn prediction models are adopted to increase the efficiency of retention campaigns by optimizing the selection of customers that are to be targeted. Cost-sensitive and causal classification methods have independently been proposed to improve the performance of classification models. The former considers the benefits and costs of correct and incorrect classifications, such as the benefit of a retained customer, whereas the latter estimates the causal effect of an action, such as a retention campaign, on the outcome of interest. This study integrates cost-sensitive and causal classification by elaborating a unifying evaluation framework. The framework encompasses a range of existing and novel performance measures for evaluating both causal and conventional classification models in a cost-sensitive as well as a cost-insensitive manner. We proof that conventional classification is a specific case of causal classification in terms of a range of performance measures when the number of actions is equal to one. The framework is shown to instantiate to application-specific cost-sensitive performance measures that have been recently proposed for evaluating customer retention and response uplift models, and allows to maximize profitability when adopting a causal classification model for optimizing decision-making. The proposed framework paves the way toward the development of cost-sensitive causal learning methods and opens a range of opportunities for improving data-driven business decision-making.

To do or not to do? Cost-sensitive causal classification with individual treatment effect estimates

“…This section formally introduces the classification and causal classification tasks and summarizes the cost-sensitive evaluation framework to assess performance of classification and causal classification models as presented in Verbeke et al (2020), upon which we extend in the following section.…”

“…A range of metrics for evaluating the performance of classification models at threshold φ have been proposed, such as accuracy, F1, sensitivity and specificity (Verbeke et al, 2020). Performance measures that assess the quality of the classification model without making an assumption about the operational threshold include, for example, the area under the receiver operating characteristic curve (AUC) and the area under the precision-recall curve (PRAUC).…”

“…The prior class probabilities in the treatment and control group are denoted by π T y and π C y , respectively. The causal confusion matrix, ĊF(τ ), summarizes the proportion of instances with a negative and positive outcome in the negative and positive treatment class, for an arbitrary causal classification threshold τ (Verbeke et al, 2020):…”

“…In this article, we build on the foundations of cost-sensitive classification (Elkan, 2001) and costsensitive causal classification (Verbeke et al, 2020), in order to formulate a cost-sensitive classification boundary for causally classifying instances in treatment classes. This boundary allows deciding which treatment to apply to an instance, as a function of the individual treatment effect and the positive outcome probability estimates, as well as the cost and benefit parameters of the problem setting, which may be defined either class-or instance-dependent.…”

Combining the Clinical and Operational Perspectives in Heterogeneous Treatment Effect Inference in Healthcare Processes

“…Previous work for healthcare solely considers purely clinical outcomes of actions. However, it has been shown in a business context that taking into account costs greatly improves total profit [5,32]. Similarly, it makes sense to account for overarching operational objectives.…”