Limited Generalization Capabilities of Autoencoders with Logistic Regression on Training Sets of Small Sizes

Potapov, Alexey; Batishcheva, Vita; Peterson, Maxim

doi:10.1007/978-3-662-44654-6_25

Cited by 3 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in some areas of the medical field, such as rare diseases, only a small dataset is available. Dimensionality reduction using AE is primarily performed via semi‐supervised learning and owing to its low risk of overlearning, it has attracted considerable attention as a suitable feature‐reduction method for DL on a small dataset 26,27 . In this study, 3D‐CCVAE performed better than the without AE model, indicating that dimensionality reduction via AE can be used to learn a DL model without increasing the overlearning risk.…”

Section: Discussionmentioning

confidence: 74%

Prognosis prediction of patients with malignant pleural mesothelioma using conditional variational autoencoder on 3D PET images and clinical data

Matsuo,

Kitajima,

Kono

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundDeep learning (DL) has been widely used for diagnosis and prognosis prediction of numerous frequently occurring diseases. Generally, DL models require large datasets to perform accurate and reliable prognosis prediction and avoid overlearning. However, prognosis prediction of rare diseases is still limited owing to the small number of cases, resulting in small datasets.PurposeThis paper proposes a multimodal DL method to predict the prognosis of patients with malignant pleural mesothelioma (MPM) with a small number of 3D positron emission tomography–computed tomography (PET/CT) images and clinical data.MethodsA 3D convolutional conditional variational autoencoder (3D‐CCVAE), which adds a 3D‐convolutional layer and conditional VAE to process 3D images, was used for dimensionality reduction of PET images. We developed a two‐step model that performs dimensionality reduction using the 3D‐CCVAE, which is resistant to overlearning. In the first step, clinical data were input to condition the model and perform dimensionality reduction of PET images, resulting in more efficient dimension reduction. In the second step, a subset of the dimensionally reduced features and clinical data were combined to predict 1‐year survival of patients using the random forest classifier. To demonstrate the usefulness of the 3D‐CCVAE, we created a model without the conditional mechanism (3D‐CVAE), one without the variational mechanism (3D‐CCAE), and one without an autoencoder (without AE), and compared their prediction results. We used PET images and clinical data of 520 patients with histologically proven MPM. The data were randomly split in a 2:1 ratio (train : test) and three‐fold cross‐validation was performed. The models were trained on the training set and evaluated based on the test set results. The area under the receiver operating characteristic curve (AUC) for all models was calculated using their 1‐year survival predictions, and the results were compared.ResultsWe obtained AUC values of 0.76 (95% confidence interval [CI], 0.72–0.80) for the 3D‐CCVAE model, 0.72 (95% CI, 0.68–0.77) for the 3D‐CVAE model, 0.70 (95% CI, 0.66–0.75) for the 3D‐CCAE model, and 0.69 (95% CI 0.65–0.74) for the without AE model. The 3D‐CCVAE model performed better than the other models (3D‐CVAE, p = 0.039; 3D‐CCAE, p = 0.0032; and without AE, p = 0.0011).ConclusionsThis study demonstrates the usefulness of the 3D‐CCVAE in multimodal DL models learned using a small number of datasets. Additionally, it shows that dimensionality reduction via AE can be used to learn a DL model without increasing the overlearning risk. Moreover, the VAE mechanism can overcome the uncertainty of the model parameters that commonly occurs for small datasets, thereby eliminating the risk of overlearning. Additionally, more efficient dimensionality reduction of PET images can be performed by providing clinical data as conditions and ignoring clinical data‐related features.

show abstract

Section: Discussionmentioning

confidence: 74%

Prognosis prediction of patients with malignant pleural mesothelioma using conditional variational autoencoder on 3D PET images and clinical data

Matsuo,

Kitajima,

Kono

et al. 2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…However, to do this, it is required to set up a fairly large data base with the possibility of monitoring the orientation of the patterns to be recognized. Taking into account our investigations, which show that deep-learning networks are limited in extracting a priori unknown invariants, as well as the limited generalizing capability of these networks on samples of small size, 17 it can be concluded that the deep-learning paradigm requires appreciable extension.…”

Section: Generalizing Capability Of Cnns During Training With Rotatiomentioning

confidence: 90%

Investigation of the generalizing capabilities of convolutional neural networks in forming rotation-invariant attributes

Malashin

Kadykov

2015

J. Opt. Technol.

View full text Add to dashboard Cite

This paper gives the results of a study of the possibilities of convolutional neural networks to generalize knowledge concerning primitive geometrical image transformations when solving pattern-recognition problems of handwritten numerals. Experiments were directed to the study of how the recognition of patterns in arbitrary orientations is affected by broadening the training sample with rotated images. Results are presented for convolutional neural networks of two architectures, showing that, to ensure rotation-invariant recognition, it is necessary for all classes of images in the entire range of rotations to be present in the training sample.

show abstract

A feasibility study of an autoencoder meta-model for improving generalization capabilities on training sets of small sizes

Potapov

Potapova

Peterson

2016

Pattern Recognition Letters

View full text Add to dashboard Cite

Limited Generalization Capabilities of Autoencoders with Logistic Regression on Training Sets of Small Sizes

Cited by 3 publications

References 12 publications

Prognosis prediction of patients with malignant pleural mesothelioma using conditional variational autoencoder on 3D PET images and clinical data

Prognosis prediction of patients with malignant pleural mesothelioma using conditional variational autoencoder on 3D PET images and clinical data

Investigation of the generalizing capabilities of convolutional neural networks in forming rotation-invariant attributes

A feasibility study of an autoencoder meta-model for improving generalization capabilities on training sets of small sizes

Contact Info

Product

Resources

About