Advantages of Hybrid Deep Learning Frameworks in Applications with Limited Data

Abstract

Recent advancements in deep learning (DL) frameworks based on deep neural networks (DNN) drastically improved accuracy in image recognition, natural language processing and other applications. The key advantage of DL is systematic approach for independent training of groups of DNN layers including unsupervised training of auto-encoders for hierarchical representation of raw input data (i.e., automatic feature selection and dimensionality reduction) and supervised re-training of several final layers in the transfer learning that compensate for data incompleteness. However, severe data limitations and/or absence of relevant problem for transfer learning can drastically reduce advantages of DNN-based DL. For example, pure data-driven auto-encoders dealing with high-dimensional input data require large amount of data for effective operation. However, hierarchical data representations can be also implemented without NN. Previously we have shown robustness of boosting-like algorithms for effective utilization of existing domain knowledge (e.g. analytical models) via discovery of compact ensembles of complementary low-complexity components. This approach can tolerate significant data incompleteness and boost accuracy of individual base models as was demonstrated in cardiac diagnostics applications. Here we argue that hybrid DL framework with auto-encoders replaced by components discovered by boosting followed by supervised NN could be more tolerant to data incompleteness compared to pure DNN-based DL. Illustrations based on cardio data from www.physionet.org are presented. The proposed framework could be utilized in many applications dealing with incomplete data including personalized medicine and rare or complex abnormalities.

Predictive Maintenance of Electromechanical Systems Using Deep Learning Algorithms: Review

“…Moreover, the enhancement of this model can contain two unsupervised, two supervised methods, and one supervised with one unsupervised [33]. For example, a multilayered perceptron is a standard of DNN are contains a group of layered before the real multi-layered perceptron layers for classification, these layers are used successfully for feature extraction and dimension reduction unsupervised before training and input data filter [34]. Figure 2 illustrates the types of DL networks.…”

Recent advances in applications of artificial intelligence in solid waste management: A review

“…This problem occurs by supplying less data to the system. These kinds of problems can be solved by using hybrid models (Gavrishchaka et al, 2018;Ullah et al, 2021a).…”

Improving the Criteria of the Investment on Stock Market Using Data Mining Techniques: The Case of S&P500 Index

“…Deep Standard DNN provides a universal framework for modelling complex and high-dimensional data. An especially attractive feature of DNN approach is the inherent capability of covering all stages of data-driven modelling (features selection, data transformation, and classification/regression) within a single framework, i.e., ideally, the practitioner can start with raw data in the domain of interest and get ready-to-use solution; besides, this deep feature hierarchy enables DNNs to achieve good performance in many tasks [16], [17].…”