Supporting AI-Explainability by Analyzing Feature Subsets in a Machine Learning Model

Plagwitz, Lucas; Brenner, Alexander; Fujarski, Michael; Varghese, Julian

doi:10.3233/shti220406

Cited by 4 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The important features contributing to the development of the combined model were evaluated using group permutation importance. [13] Among the top five important features, three clinical parameters, namely “age,” “radiation dose,” and “preoperative KPS,” were included, along with the deep imaging features extracted from MRI, “postoperative mask image,” and “preoperative mask image.” Analysis of grouped permutation importance supported the idea that MRI features contributed to model development and improved model performance. We consistently observed an improvement in prediction performance when incorporating clinical parameters and MRI features.…”

Section: Discussionmentioning

confidence: 99%

Section: Discussion Improved Performance When Incorporating Multimoda...mentioning

confidence: 99%

“…To improve the understanding, trust, and verification of the model predictions, grouped permutation importance was applied. [13] Grouped permutation importance quantifies the feature contribution, thus providing an interpretable relationship between the incorporated features and the model prediction. Fig 4D illustrates the importance of the top five relative features for the multimodal model to predict a 6-month KPS score of <70.…”

Section: Model Interpretability Analysismentioning

confidence: 99%

“…Grouped permutation feature importance was utilized to determine the feature importance in the neural network-based prediction model. [13] In this analysis, features were optionally grouped into expert-defined subgroups, and a systematic assessment of their importance was conducted. In this study, during the feature extraction process, 196 MRI features were generated from preoperative and postoperative MRI imaging.…”

Section: Model Interpretabilitymentioning

confidence: 99%

“…To improve the understanding, trust, and verification of the model predictions, grouped permutation importance was applied. [13] Grouped permutation importance quantifies the feature contribution, thus providing an interpretable relationship between the incorporated features and the . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.…”

Section: Model Interpretability Analysismentioning

confidence: 99%

See 4 more Smart Citations

Postoperative Karnofsky performance status prediction in patients withIDHwild-type glioblastoma: a multimodal approach integrating clinical and deep imaging features

Sasagasako,

Ueda,

Mineharu

et al. 2024

Preprint

View full text Add to dashboard Cite

Background and Purpose: Glioblastoma is a highly aggressive brain tumor with limited survival that poses challenges in predicting patient outcomes. The Karnofsky Performance Status (KPS) score is a valuable tool for assessing patient functionality and contributes to the stratification of patients with poor prognoses. This study aimed to develop a 6-month postoperative Karnofsky Performance Status (KPS) prediction model by combining clinical data with deep learning-based image features from pre- and postoperative MRI scans, offering enhanced personalized care for glioblastoma patients. Materials and Methods: Using 1,476 MRI datasets from the Brain Tumor Segmentation Challenge 2020 public database, we pretrained two variational autoencoders (VAEs). Imaging features from the latent spaces of the VAEs were used for KPS prediction. Neural network-based KPS prediction models were developed to predict scores below 70 at 6 months postoperatively. In this retrospective single-center analysis, we incorporated clinical parameters and pre- and postoperative MRI images from 150 newly diagnosed IDH wild-type glioblastoma, divided into training (100 patients) and test (50 patients) sets. In training set, the performance of these models was evaluated using the area under the curve (AUC), calculated through fivefold cross-validation repeated 10 times. The performance of these models in the training set was evaluated using the area under the curve (AUC) from a fivefold cross-validation repeated 10 times. The final evaluation of the developed models assessed in the test set. Results: Among the 150 patients, 61 had 6-month postoperative KPS scores below 70 and 89 scored 70 or higher. We developed three models: a clinical-based model, an MRI-based model, and a multimodal model that incorporated both clinical parameters and MRI features. In the training set, the mean AUC was 0.785±0.051 for the multimodal model, which was significantly higher than the clinical-based model (0.716±0.059, P=0.038) using only clinical parameters and MRI-based model (0.651±0.028, P<0.001) using only MRI features. In the test set, the multimodal model achieved an AUC of 0.810, outperforming the clinical-based (0.670) and MRI-based (0.650) models. Conclusion: The integration of MRI features extracted from VAEs with clinical parameters in the multimodal model substantially enhanced KPS prediction performance. This approach has the potential to improve prognostic prediction, paving the way for more personalized and effective treatments for patients with glioblastoma. Abbreviations: KPS, Karnofsky performance status. IDH, isocitrate dehydrogenase. VAE, variational autoencoder. BraTS, Brain Tumor Segmentation challenge

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussion Improved Performance When Incorporating Multimoda...mentioning

confidence: 99%

Section: Model Interpretability Analysismentioning

confidence: 99%

Section: Model Interpretabilitymentioning

confidence: 99%

“…To improve the understanding, trust, and verification of the model predictions, grouped permutation importance was applied. [13] Grouped permutation importance quantifies the feature contribution, thus providing an interpretable relationship between the incorporated features and the . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.…”

Section: Model Interpretability Analysismentioning

confidence: 99%

See 3 more Smart Citations

Postoperative Karnofsky performance status prediction in patients withIDHwild-type glioblastoma: a multimodal approach integrating clinical and deep imaging features

Sasagasako,

Ueda,

Mineharu

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Machine Learning in the Parkinson’s disease smartwatch (PADS) dataset

Varghese,

Brenner,

Fujarski

et al. 2024

npj Parkinsons Dis.

View full text Add to dashboard Cite

The utilisation of smart devices, such as smartwatches and smartphones, in the field of movement disorders research has gained significant attention. However, the absence of a comprehensive dataset with movement data and clinical annotations, encompassing a wide range of movement disorders including Parkinson’s disease (PD) and its differential diagnoses (DD), presents a significant gap. The availability of such a dataset is crucial for the development of reliable machine learning (ML) models on smart devices, enabling the detection of diseases and monitoring of treatment efficacy in a home-based setting. We conducted a three-year cross-sectional study at a large tertiary care hospital. A multi-modal smartphone app integrated electronic questionnaires and smartwatch measures during an interactive assessment designed by neurologists to provoke subtle changes in movement pathologies. We captured over 5000 clinical assessment steps from 504 participants, including PD, DD, and healthy controls (HC). After age-matching, an integrative ML approach combining classical signal processing and advanced deep learning techniques was implemented and cross-validated. The models achieved an average balanced accuracy of 91.16% in the classification PD vs. HC, while PD vs. DD scored 72.42%. The numbers suggest promising performance while distinguishing similar disorders remains challenging. The extensive annotations, including details on demographics, medical history, symptoms, and movement steps, provide a comprehensive database to ML techniques and encourage further investigations into phenotypical biomarkers related to movement disorders.

show abstract

Feature Selection for Automated QoE Prediction

Kikuzuki,

Mashhadi,

et al. 2023

2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)

View full text Add to dashboard Cite

With the huge number of broadband users, automated network management becomes of huge interest to service providers. A major challenge is automated monitoring of user Quality of Experience (QoE), where Artificial Intelligence (AI) and Machine Learning (ML) models provide powerful tools to predict user QoE from basic protocol indicators such as Round Trip Time (RTT), retransmission rate, etc. In this paper, we introduce an effective feature selection method along with the corresponding classification algorithms to address this challenge. The simulation results show a prediction accuracy of 78% on the benchmark ITU ML5G-PS-012 dataset, improving 11% over the state-of-the-art result whilst reducing the model complexity at the same time. Moreover, we show that the local area network round trip time (LAN RTT) value during daytime and midweek plays the most prominent factor affecting the user QoE.

show abstract

Supporting AI-Explainability by Analyzing Feature Subsets in a Machine Learning Model

Cited by 4 publications

References 0 publications

Postoperative Karnofsky performance status prediction in patients withIDHwild-type glioblastoma: a multimodal approach integrating clinical and deep imaging features

Postoperative Karnofsky performance status prediction in patients withIDHwild-type glioblastoma: a multimodal approach integrating clinical and deep imaging features

Machine Learning in the Parkinson’s disease smartwatch (PADS) dataset

Feature Selection for Automated QoE Prediction

Contact Info

Product

Resources

About