Benchmark of Encoders of Nominal Features for Regression

Seca, Diogo; Mendes-Moreira, João

doi:10.1007/978-3-030-72657-7_14

Cited by 2 publications

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As those studies only consider gradient boosting and a limited amount of datasets, it is unclear whether results generalize to other datasets and ML algorithms. A recent study (15 datasets) found good performance of target encoding, but they only investigated categorical features in regression settings (Seca and Mendes-Moreira 2021). Several other publications studied encoding text data based on similarity (Cerda et al 2018;Cerda and Varoquaux 2020) and employed indicator or target encoding as baselines.…”

Section: Categorical Encoding Benchmarksmentioning

confidence: 99%

Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features

Pargent

Pfisterer²,

Thomas³

et al. 2022

Comput Stat

View full text Add to dashboard Cite

Since most machine learning (ML) algorithms are designed for numerical inputs, efficiently encoding categorical variables is a crucial aspect in data analysis. A common problem are high cardinality features, i.e. unordered categorical predictor variables with a high number of levels. We study techniques that yield numeric representations of categorical variables which can then be used in subsequent ML applications. We focus on the impact of these techniques on a subsequent algorithm’s predictive performance, and—if possible—derive best practices on when to use which technique. We conducted a large-scale benchmark experiment, where we compared different encoding strategies together with five ML algorithms (lasso, random forest, gradient boosting, k-nearest neighbors, support vector machine) using datasets from regression, binary- and multiclass–classification settings. In our study, regularized versions of target encoding (i.e. using target predictions based on the feature levels in the training set as a new numerical feature) consistently provided the best results. Traditionally widely used encodings that make unreasonable assumptions to map levels to integers (e.g. integer encoding) or to reduce the number of levels (possibly based on target information, e.g. leaf encoding) before creating binary indicator variables (one-hot or dummy encoding) were not as effective in comparison.

show abstract

Section: Categorical Encoding Benchmarksmentioning

confidence: 99%