A Systematic Review on Data Scarcity Problem in Deep Learning: Solution and Applications

Abstract

Recent advancements in deep learning architecture have increased its utility in real-life applications. Deep learning models require a large amount of data to train the model. In many application domains, there is a limited set of data available for training neural networks as collecting new data is either not feasible or requires more resources such as in marketing, computer vision, and medical science. These models require a large amount of data to avoid the problem of overfitting. One of the data space solutions to the problem of limited data is data augmentation. The purpose of this study focuses on various data augmentation techniques that can be used to further improve the accuracy of a neural network. This saves the cost and time consumption required to collect new data for the training of deep neural networks by augmenting available data. This also regularizes the model and improves its capability of generalization. The need for large datasets in different fields such as computer vision, natural language processing, security and healthcare is also covered in this survey paper. The goal of this paper is to provide a comprehensive survey of recent advancements in data augmentation techniques and their application in various domains.

Real-world data mining meets clinical practice: Research challenges and perspective

“…The number of patients involved can be limited even in multicentric studies; as in Orsini et al (2017) and Ferrari et al (2020a), where the few patients were unevenly enrolled across Modena, Hong Kong, and Sydney hospitals. Scarcity and sparsity are not always in contrast, and methods exist to limit both their effects (Bansal et al, 2021).…”

Mislabeled learning for psychiatric disorder detection

“…The deep learning models generally need a large amount of data in training to build sophisticated prediction functions, but they are not good for high-dimensional data with only a small number of samples. This is mainly because almost all deep learning models suffer from the data scarcity problem, i.e., deep learning may have poor performance for datasets with only a limited number of observations [22].…”

“…On the other hand, the existing mislabeled learning techniques may not be able to apply to the high-dimensional SNP data in out context because they are mainly developed for lowdimensional data in which a large number of samples are available. For example, adding a noise filtering in deep neural networks (DNN) may filter samples with corrupted labels, but it can't apply to highdimensional data because the filtering scheme can be too luxury to implement for a dataset only with limited number of samples [1,22]. Thus, it is necessary to seek an effective mislabeling correction technique, which acts as the relabeling function 𝑓 ) , to tackle the problem, to retrieve the most likely ground truth for each sample.…”

“…It makes it possible to handle mislabeled learning problems for small-sized datasets. The small-sized datasets along with mislabeled information across data would fail almost all deep learning models because of the data scarcity issue [36]. Therefore, the proposed technology is especially valuable not only for psychiatry but also for most biomedical fields.…”

“…Furthermore, the proposed mislabeled learning algorithm does not require a large number of samples and works well for the ‘small-sample data’, the datasets with a limited number of samples. Therefore, it shows a good advantage in handling the sample scarcity issue in learning over almost all mislabeled learning peers [22].…”