Evaluating the Impact of Categorical Data Encoding and Scaling on Neural Network Classification Performance: The Case of Repeat Consumption of Identical Cultural Goods

Fitkov-Norris, Elena; Vahid, Samireh; Hand, Chris

doi:10.1007/978-3-642-32909-8_35

Cited by 18 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, note that we often lose information with respect to the original data when applying preprocessing methods for deep neural networks, leading to a reduction in predictive performance (Fitkov-Norris et al, 2012). 4.…”

Section: Missing or Complex Irregular Spatial Dependenciesmentioning

confidence: 99%

Deep Neural Networks and Tabular Data: A Survey

Borisov¹,

Leemann²,

Seßler³

et al. 2021

Preprint

View full text Add to dashboard Cite

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous data sets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their application to modeling tabular data (inference or generation) remains highly challenging. This work provides an overview of state-of-the-art deep learning methods for tabular data. We start by categorizing them into three groups: data transformations, specialized architectures, and regularization models. We then provide a comprehensive overview of the main approaches in each group. A discussion of deep learning approaches for generating tabular data is complemented by strategies for explaining deep models on tabular data. Our primary contribution is to address the main research streams and existing methodologies in this area, while highlighting relevant challenges and open research questions. To the best of our knowledge, this is the first in-depth look at deep learning approaches for tabular data. This work can serve as a valuable starting point and guide for researchers and practitioners interested in deep learning with tabular data.

show abstract

Section: Missing or Complex Irregular Spatial Dependenciesmentioning

confidence: 99%

Deep Neural Networks and Tabular Data: A Survey

Borisov¹,

Leemann²,

Seßler³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We compare our methods to several baselines. The first set of baselines consists of different encoding schemes for ordinal data (Kurczynski 1970;Boriah, Chandola, and Kumar 2008;Fitkov-Norris, Vahid, and Hand 2012), and the remaining ones are methods specially designed for measuring similarities on ordinal data (Podani 1999). All these baseline algorithms are summarized below:…”

Section: Methodsmentioning

confidence: 99%

Metric Learning for Ordinal Data

Shi

Sha

2016

AAAI

View full text Add to dashboard Cite

A large amount of ordinal-valued data exist in many domains, including medical and health science, social science, economics, political science, etc. Unlike image and speech datasets of real-valued data, learning with ordinal variables (i.e., features) presents unique challenges. In particular, the nominal differences between those feature values, which are just ranks, do not necessarily correspond to the real distances between the corresponding categories. Given their wide existence, it is imperative to develop machine learning algorithms that specifically address the need to model and infer with such data. In this paper, we present a novel metric learning algorithm that takes into consideration the nature of ordinal data. Our approach treats ordinal values as latent variables in intervals. Our algorithm then learns what those intervals are as well as distance metrics to measure distances between latent variables in those intervals. We derive the corresponding optimization algorithm and demonstrate how that can be solved effectively. Experimental results show that the proposed approach significantly improves baselines that do not explicitly model ordinal features.

show abstract

“…Even for variables with an inherent order, it leads to many similar binary encodings (i.e. encodings that agree for many of the bits) for semantically very different categorical values (Fitkov-Norris et al, 2012).…”

Section: Literature Reviewmentioning

confidence: 99%

Hierarchy-based semantic embeddings for single-valued & multi-valued categorical variables

Mumtaz

Giese

2021

J Intell Inf Syst

View full text Add to dashboard Cite

In low-resource domains, it is challenging to achieve good performance using existing machine learning methods due to a lack of training data and mixed data types (numeric and categorical). In particular, categorical variables with high cardinality pose a challenge to machine learning tasks such as classification and regression because training requires sufficiently many data points for the possible values of each variable. Since interpolation is not possible, nothing can be learned for values not seen in the training set. This paper presents a method that uses prior knowledge of the application domain to support machine learning in cases with insufficient data. We propose to address this challenge by using embeddings for categorical variables that are based on an explicit representation of domain knowledge (KR), namely a hierarchy of concepts. Our approach is to 1. define a semantic similarity measure between categories, based on the hierarchy—we propose a purely hierarchy-based measure, but other similarity measures from the literature can be used—and 2. use that similarity measure to define a modified one-hot encoding. We propose two embedding schemes for single-valued and multi-valued categorical data. We perform experiments on three different use cases. We first compare existing similarity approaches with our approach on a word pair similarity use case. This is followed by creating word embeddings using different similarity approaches. A comparison with existing methods such as Google, Word2Vec and GloVe embeddings on several benchmarks shows better performance on concept categorisation tasks when using knowledge-based embeddings. The third use case uses a medical dataset to compare the performance of semantic-based embeddings and standard binary encodings. Significant improvement in performance of the downstream classification tasks is achieved by using semantic information.

show abstract

Evaluating the Impact of Categorical Data Encoding and Scaling on Neural Network Classification Performance: The Case of Repeat Consumption of Identical Cultural Goods

Cited by 18 publications

References 11 publications

Deep Neural Networks and Tabular Data: A Survey

Deep Neural Networks and Tabular Data: A Survey

Metric Learning for Ordinal Data

Hierarchy-based semantic embeddings for single-valued & multi-valued categorical variables

Contact Info

Product

Resources

About