Everything has its price: Foundations of cost-sensitive machine learning and its application in psychology.

Abstract

Psychology has seen an increase in the use of machine learning (ML) methods. In many applications, observations are classified into one of two groups (binary classification). Off-the-shelf classification algorithms assume that the costs of a misclassification (false positive or false negative) are equal. Because this is often not reasonable (e.g., in clinical psychology), cost-sensitive machine learning (CSL) methods can take different cost ratios into account. We present the mathematical foundations and introduce a taxonomy of the most commonly used CSL methods, before demonstrating their application and usefulness on psychological data, that is, the drug consumption data set (N = 1, 885) from the University of California Irvine ML Repository. In our example, all demonstrated CSL methods noticeably reduced mean misclassification costs compared to regular ML algorithms. We discuss the necessity for researchers to perform small benchmarks of CSL methods for their own practical application. Thus, our open materials provide R code, demonstrating how CSL methods can be applied within the mlr3 framework (https://osf.io/cvks7/).

Never miss a beep: Using mobile sensing to investigate (non-)compliance in experience sampling studies

“…Regarding preprocessing, we also tested approaches to account for the class imbalance in our target variable such as the assignment of class-dependent weights or oversampling (Sterner et al., 2023 ) and model-specific hyperparameter tuning (e.g., lambda or mtry for elastic net and random forest, respectively). We reran the models without the described exclusion of extreme outliers (±4 SD of the mean) for which results are provided in the online materials in the OSF repository.…”

A Fake Profile Detection Model Using Multistage Stacked Ensemble Classification

“…In the case of fake profile detection, fake profiles are regarded as positive samples, and genuine profiles as negative samples. Therefore, the misclassification cost of belonging to the minority class should be higher than that belonging to the majority [5,25]. In cost-sensitive learning, the primary idea is to train the classifier to have a minimum classification error.…”