Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

Nemani, Venkat; Biggio, Luca; Huan, Xun; Hu, Zhen; Fink, Olga; Tran, Anh; Wang, Yan; Zhang, Xiaoge; Hu, Chao

doi:10.1016/j.ymssp.2023.110796

Cited by 49 publications

(17 citation statements)

References 227 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Addressing these concerns, Gaussian Process Regression (GPR) has been extensively applied in high-stakes decision applications such as aviation design and healthcare. This is due to the intrinsic property of providing distance-aware uncertainty estimates [5]. Using a GPR, a prediction and the associated uncertainty are derived by conditioning a prior with the training data [6,7].…”

Section: Data and Model Selectionmentioning

confidence: 99%

AI Driven Experiment Calibration and Control

Britton,

Bedwell,

Chawhan

et al. 2024

EPJ Web of Conf.

View full text Add to dashboard Cite

One critical step on the path from data taking to physics analysis is calibration. For many experiments this step is both time consuming and computationally expensive. The AI Experimental Calibration and Control project seeks to address these issues, starting first with the GlueX Central Drift Chamber (CDC). We demonstrate the ability of a Gaussian Process to estimate the gain correction factor (GCF) of the GlueX CDC accurately, and also the uncertainty of this estimate. Using the estimated GCF, the developed system infers a new high voltage (HV) setting that stabilizes the GCF in the face of changing environmental conditions. This happens in near real time during data taking and produces data which are already approximately gain-calibrated, eliminating the cost of performing those calibrations which vary ±15% with fixed HV. We also demonstrate an implementation of an uncertainty aware system which exploits a key feature of a Gaussian process.

show abstract

Section: Data and Model Selectionmentioning

confidence: 99%

AI Driven Experiment Calibration and Control

Britton,

Bedwell,

Chawhan

et al. 2024

EPJ Web of Conf.

View full text Add to dashboard Cite

show abstract

“…These out-of-training-distribution test samples are often called out-of-distribution (OOD) samples. A solution to the conflict between what is needed (i.e., big data) and what is available (i.e., small data) is quantifying the predictive uncertainty through probabilistic deep learning 34 . The uncertainty estimate could serve as a proxy for model confidence, i.e., how confident this model is when making a prediction.…”

Section: Standard ML Pipelinementioning

confidence: 99%

“…Our aim is to present easily digestible materials, particularly for newcomers in this field, such as fresh Ph.D. students eager to https://doi.org/10.1038/s44296-024-00011-1 grasp the fundamentals of probabilistic battery diagnostics and prognostics. We also note that there exists a broader spectrum of probabilistic ML methods beyond those discussed in this paper (e.g., see Nemani et al 34 ); we aim in this paper to highlight a select few that have been most prominently used in, and generally applicable, to the type of problems encountered in battery diagnostics and prognostics.…”

Section: Probabilistic ML Techniques and Their Applications To Batter...mentioning

confidence: 99%

“…Selecting a probabilistic ML algorithm requires assessing several criteria to ensure the algorithm's suitability for a specific use case. Several key criteria are listed as follows (see Nemani et al 34 for further details): (1) prediction accuracy (e.g., evaluated by comparing mean predictions with ground truth on a validation dataset spit out of a training dataset), (2) quality of uncertainty quantification (i.e., the algorithm's ability to produce accurate uncertainty estimates), (3) computational efficiency (an important factor in applications where real-time or near-real-time diagnostics/prognostics may be required), ( 4) scalability (i.e., the algorithm's ability to train models based on large volumes of datasets, i.e., in the big data regime), and ( 5) robustness (i.e., the algorithm's ability to maintain performance in the presence of outliers, high noise, and adversarial variations in the input data). These criteria may become conflicting objectives that need to be weighed based on the needs and wants of a specific diagnostic/prognostic use case.…”

Section: Probabilistic ML Techniques and Their Applications To Batter...mentioning

confidence: 99%

“…Despite these significant research efforts on battery health diagnostics and prognostics, most ML-focused works have yet to incorporate uncertainty quantification systematically. Here, "uncertainty" refers to the predictive uncertainty of an ML model, such as a neural network, for a training/ test sample point that is ideally associated with how confident the model is when predicting at this point 34 . The idea is that an ML model does not simply produce an output (e.g., an estimate of a cell's SOH indicator); it also estimates the uncertainty associated with this prediction to the most accurate extent possible.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic machine learning for battery health diagnostics and prognostics—review and perspectives

Thelen,

Huan,

Paulson

et al. 2024

npj Mater. Sustain.

Self Cite

View full text Add to dashboard Cite

Diagnosing lithium-ion battery health and predicting future degradation is essential for driving design improvements in the laboratory and ensuring safe and reliable operation over a product’s expected lifetime. However, accurate battery health diagnostics and prognostics is challenging due to the unavoidable influence of cell-to-cell manufacturing variability and time-varying operating circumstances experienced in the field. Machine learning approaches informed by simulation, experiment, and field data show enormous promise to predict the evolution of battery health with use; however, until recently, the research community has focused on deterministic modeling methods, largely ignoring the cell-to-cell performance and aging variability inherent to all batteries. To truly make informed decisions regarding battery design in the lab or control strategies for the field, it is critical to characterize the uncertainty in a model’s predictions. After providing an overview of lithium-ion battery degradation, this paper reviews the current state-of-the-art probabilistic machine learning models for health diagnostics and prognostics. Details of the various methods, their advantages, and limitations are discussed in detail with a primary focus on probabilistic machine learning and uncertainty quantification. Last, future trends and opportunities for research and development are discussed.

show abstract

Modeling of Gaseous Reduction of Iron Oxide Pellets Using Machine Learning Algorithms, Explainable Artificial Intelligence, and Hyperparameter Optimization Techniques

Hosseinzadeh,

Kasiri,

Rezaei

2024

steel research int.

View full text Add to dashboard Cite

In this study, a novel application of machine learning (ML) is introduced to pellet modeling in the intricate non‐catalytic gas–solid reaction of direct reduction of iron oxide in the steel industry. Twenty ML models are developed using four algorithms: multilayer perceptron neural networks (MLPNN), radial basis function neural network (RBFNN), support vector regression, and random forest (RF). Hyperparameter optimization is conducted using Bayesian algorithms, random search, and grid search. The optimum model achieves a mean squared error test of 0.0052 with random RF for the larger dataset (872 samples), while smaller datasets (132, 225, and 242 samples) produce optimum models with MLPNN and RBFNN. Hyperparameters vary between the larger datasets and the smaller datasets. The models offer insight into the complex interactions among variables, including time, temperature, gas composition, hematite composition, pellet radius, and initial pellet porosity, influencing the metallization degree. In this study, the significant role of time and temperature is emphasized, as revealed by explainable artificial intelligence using Shapley additive explanation analysis that utilizes the game theory, and the effects of pellet modeling parameters are elucidated through 3D plots, particularly highlighting the impact of changing H2/CO proportion on metallization degree and carbon deposit.

show abstract

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

Cited by 49 publications

References 227 publications

AI Driven Experiment Calibration and Control

AI Driven Experiment Calibration and Control

Probabilistic machine learning for battery health diagnostics and prognostics—review and perspectives

Modeling of Gaseous Reduction of Iron Oxide Pellets Using Machine Learning Algorithms, Explainable Artificial Intelligence, and Hyperparameter Optimization Techniques

Contact Info

Product

Resources

About