Synergy of physics-based reasoning and machine learning in biomedical applications: towards unlimited deep learning with limited data

Abstract

Technological advancements enable collecting vast data, i. e., Big Data, in science and industry including biomedical field. Increased computational power allows expedient analysis of collected data using statistical and machine-learning approaches. Historical data incompleteness problem and curse of dimensionality diminish practical value of pure data-driven approaches, especially in biomedicine. Advancements in deep learning (DL) frameworks based on deep neural networks (DNN) improved accuracy in image recognition, natural language processing, and other applications yet severe data limitations and/or absence of transferlearning-relevant problems drastically reduce advantages of DNN-based DL. Our earlier works demonstrate that hierarchical data representation can be alternatively implemented without NN, using boosting-like algorithms for utilization of existing domain knowledge, tolerating significant data incompleteness, and boosting accuracy of low-complexity models within the classifier ensemble, as illustrated in physiological-data analysis. Beyond obvious use in initial-factor selection, existing simplified models are effectively employed for generation of realistic synthetic data for later DNN pre-training. We review existing machine learning approaches, focusing on limitations caused by trainingdata incompleteness. We outline our hybrid framework that leverages existing domain-expert models/knowledge, boosting-like model combination, DNN-based DL and other machine learning algorithms for drastic reduction of training-data requirements. Applying this framework is illustrated in context of analyzing physiological data.

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

“…Data without any consideration of the dynamics of the system [58] empowers the ML methods to be applicable to different types of systems. However, data can never be complete because of the discrete nature of the input parameters and the output variables of the system [111]. Irrespective of whether data acquisition is expensive or not, data can never fill the state space.…”

“…That simplifies the model of the system by disregarding the impact of those dynamics. That makes the MB incomplete and inaccurate [111]. For example, the impact of wind drag, energy lost due to the non-elastic nature of collisions, the elasticity of the material of the ball, the buoyancy of the medium is often ignored in case of the bouncing ball.…”

“…These aspects of the system can only be brought into the model using MB. Hence, it is sometimes prudent to complement the ML with MB [58], [111], [116]. By combining them, one can actually recover a complete model of the CPS.…”

“…CPS has opened up the gates for the flood of big data in every imaginable domain. Big data is accompanied by high dimensional input data [111]. When the dimensionality of the data is very high, the useful data is actually embedded in a low dimensional manifold.…”

Understanding ML driven HPC: Applications and Infrastructure

“…MLAutoTuning can be applied at multiple distinct points, and can be used for a range of tuning and optimization objectives. For example: (i) mix of performance and quality of results using parameters provided by learning network [4], [25]- [28]; (ii) choose the best set of "computation defining parameters" to achieve some goal such as providing the most efficient training set with defining parameters spread well over the relevant phase space [29], [30]; (iii) tuning model parameters to optimize model outputs to available empirical data [31]- [34].…”