Performance Metrics for Artificial Intelligence (AI) Algorithms Adopted in Prognostics and Health Management (PHM) of Mechanical Systems

Ochella, Sunday; Shafiee, Mahmood

doi:10.1088/1742-6596/1828/1/012005

Cited by 7 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relative prediction performance of the proposed algorithms compared to other variants of deep learning methods can be compared quantitatively using three accuracy metrics or indexes including the mean average error/mean absolute percentage error (MAE/MAPE), mean squared error (MSE), and root mean squared error (RMSE), as given in Equations ( 17)- (19). The MAPE and RMSE are preferred metrics due to their ability to punish large errors with square roots [31].…”

Section: Data Preprocessing and Evaluation Metricsmentioning

confidence: 99%

Long-Term Lifetime Prediction of Power MOSFET Devices Based on LSTM and GRU Algorithms

Ibrahim

Abbas²,

Waseem

et al. 2023

Mathematics

View full text Add to dashboard Cite

Predicting the long-term lifetime of power MOSFET devices plays a central role in the prevention of unprecedented failures for power MOSFETs used in safety-critical applications. The various traditional model-based approaches and statistical and filtering algorithms for prognostics have limitations in terms of handling the dynamic nature of failure precursor degradation data for these devices. In this paper, a prognostic model based on LSTM and GRU is developed that aims at estimating the long-term lifetime of discrete power MOSFETs using dominant failure precursor degradation data. An accelerated power cycling test has been designed and executed to collect failure precursor data. For this purpose, commercially available power MOSFETs passed through power cycling tests at different temperature swing conditions and potential failure precursor data were collected using an automated curve tracer after certain intervals. The on-state resistance degradation data identified as one of the dominant failure precursors and potential aging precursors has been analyzed using RNN, LSTM, and GRU-based algorithms. The LSTM and GRU models have been found to be superior compared to RNN, with MAPE of 0.9%, 0.78%, and 1.72% for MOSFET 1; 0.90%, 0.66%, and 0.6% for MOSFET 5; and 1.05%, 0.9%, and 0.78%, for MOSFET 9, respectively, predicted at 40,000 cycles. In addition, the robustness of these methods is examined using training data at 24,000 and 54,000 cycles of starting points and is able to predict the long-term lifetime accurately, as evaluated by MAPE, MSE, and RMSE metrics. In general, the prediction results showed that the prognostics algorithms developed were trained to provide effective, accurate, and useful lifetime predictions and were found to address the reliability concerns of power MOSFET devices for practical applications.

show abstract

Section: Data Preprocessing and Evaluation Metricsmentioning

confidence: 99%

Long-Term Lifetime Prediction of Power MOSFET Devices Based on LSTM and GRU Algorithms

Ibrahim

Abbas²,

Waseem

et al. 2023

Mathematics

View full text Add to dashboard Cite

show abstract

“…Some of the metrics that exist are regression metrics, classification accuracy, confusion matrix of ground-truth labels versus model predictions, precision, recall, F1-score, etc. [41] [42] [43].…”

Section: Performancementioning

confidence: 99%

Artificial Intelligence Nomenclature Identified From Delphi Study on Key Issues Related to Trust and Barriers to Adoption for Autonomous Systems

Doyle,

Tucci,

Zhu

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>The rapid integration of artificial intelligence across traditional research domains has generated an amalgamation of nomenclature. As cross-discipline teams work together on complex machine learning challenges, finding a consensus of basic definitions in the literature is a more fundamental problem. As a step in the Delphi process to define issues with trust and barriers to the adoption of autonomous systems, our study first collected and ranked the top concerns from a panel of international experts from the fields of engineering, computer science, medicine, aerospace, and defence, with experience working with artificial intelligence. This document presents a summary of the literature definitions for nomenclature derived from expert feedback.</p>

show abstract

“…The prevalent issue of class imbalance, often due to limited medical data [14], can introduce statistical biases leading to result misinterpretations [10]. This imbalance allows larger classes to disproportionately influence model predictions [15], affecting model performance as highlighted in various studies [16,17]. Therefore, choosing appropriate metrics is crucial to accurately reflect model performance, especially in AIdriven models [17].…”

Section: Introductionmentioning

confidence: 99%

“…Studies such as that of [16] underscore the importance of metrics as confidence indicators in algorithms and methodologies. However, ref.…”

Section: Introductionmentioning

confidence: 99%

“…The confusion matrix is not only used to measure the efficacy of models in the analysis of clinical pathologies but has also been extensively examined in numerous studies that make direct use of its metrics or combinations thereof. Authors such as [10,16,17,32,33] highlight various aspects of performance evaluation, focusing on this metric among others. They share a common view on its applications and the limitations it presents in contexts with imbalanced clinical data.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Unraveling the Impact of Class Imbalance on Deep-Learning Models for Medical Image Classification

Hellín,

Olmedo,

Valledor

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

The field of image analysis with artificial intelligence has grown exponentially thanks to the development of neural networks. One of its most promising areas is medical diagnosis through lung X-rays, which are crucial for diseases like pneumonia, which can be mistaken for other conditions. Despite medical expertise, precise diagnosis is challenging, and this is where well-trained algorithms can assist. However, working with medical images presents challenges, especially when datasets are limited and unbalanced. Strategies to balance these classes have been explored, but understanding their local impact and how they affect model evaluation is still lacking. This work aims to analyze how a class imbalance in a dataset can significantly influence the informativeness of metrics used to evaluate predictions. It demonstrates that class separation in a dataset impacts trained models and is a strategy deserving more attention in future research. To achieve these goals, classification models using artificial and deep neural networks implemented in the R environment are developed. These models are trained using a set of publicly available images related to lung pathologies. All results are validated using metrics obtained from the confusion matrix to verify the impact of data imbalance on the performance of medical diagnostic models. The results raise questions about the procedures used to group classes in many studies, aiming to achieve class balance in imbalanced data and open new avenues for future research to investigate the impact of class separation in datasets with clinical pathologies.

show abstract

Performance Metrics for Artificial Intelligence (AI) Algorithms Adopted in Prognostics and Health Management (PHM) of Mechanical Systems

Cited by 7 publications

References 5 publications

Long-Term Lifetime Prediction of Power MOSFET Devices Based on LSTM and GRU Algorithms

Long-Term Lifetime Prediction of Power MOSFET Devices Based on LSTM and GRU Algorithms

Artificial Intelligence Nomenclature Identified From Delphi Study on Key Issues Related to Trust and Barriers to Adoption for Autonomous Systems

Unraveling the Impact of Class Imbalance on Deep-Learning Models for Medical Image Classification

Contact Info

Product

Resources

About