Toward Understanding Acceleration-based Activity Recognition Neural Networks with Activation Maximization

Yoshimura, Naoya; Maekawa, Taira; Hara, Takahiro

doi:10.1109/ijcnn52387.2021.9533888

Cited by 8 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2) Similar to approaches in spectral importance, activation maximization [56] can be applied to identify key waveforms [16]. Activation maximization approaches have been applied to other types of time-series classification [69], [70]. The methods in these two studies optimize the content of a sample in the time domain and are effective for short time-series (i.e., around 30 time points long).…”

Section: Taxonomy Of Explainability Methods For Deep Learning Models ...mentioning

confidence: 99%

“…This led to one study optimizing the spectral content of a sample to create waveforms [16]. This approach obtains more realistic waveforms than the approaches shown in other domains for shorter time-series [69], [70] but still leaves room for improvement. (3) Lastly, model visualization approaches that provide insights into important spectral features can also identify the importance of waveforms when paired with perturbation of model filters [5], [8], [40], [67].…”

Section: Taxonomy Of Explainability Methods For Deep Learning Models ...mentioning

confidence: 99%

See 1 more Smart Citation

A Framework for Systematically Evaluating the Representations Learned by A Deep Learning Classifier from Raw Multi-Channel Electroencephalogram Data

Ellis

Sattiraju

Miller

et al. 2023

Preprint

View full text Add to dashboard Cite

The application of deep learning methods to raw electroencephalogram (EEG) data is growing increasingly common. While these methods offer the possibility of improved performance relative to other approaches applied to manually engineered features, they also present the problem of reduced explainability. As such, a number of studies have sought to provide explainability methods uniquely adapted to the domain of deep learning-based raw EEG classification. In this study, we present a taxonomy of those methods, identifying existing approaches that provide insight into spatial, spectral, and temporal features. We then present a novel framework consisting of a series of explainability approaches for insight into classifiers trained on raw EEG data. Our framework provides spatial, spectral, and temporal explanations similar to existing approaches. However, it also, to the best of our knowledge, proposes the first explainability approaches for insight into spatial and spatio-spectral interactions in EEG. This is particularly important given the frequent use and well-characterized importance of EEG connectivity measures for neurological and neuropsychiatric disorder analysis. We demonstrate our proposed framework within the context of automated major depressive disorder (MDD) diagnosis, training a high performing one-dimensional convolutional neural network with a robust cross-validation approach on a publicly available dataset. We identify interactions between central electrodes and other electrodes and identify differences in frontal theta, beta, and gamma low between healthy controls and individuals with MDD. Our study represents a significant step forward for the field of deep learning-based raw EEG classification, providing new capabilities in interaction explainability and providing direction for future innovations through our proposed taxonomy.

show abstract

Section: Taxonomy Of Explainability Methods For Deep Learning Models ...mentioning

confidence: 99%

Section: Taxonomy Of Explainability Methods For Deep Learning Models ...mentioning

confidence: 99%

A Framework for Systematically Evaluating the Representations Learned by A Deep Learning Classifier from Raw Multi-Channel Electroencephalogram Data

Ellis

Sattiraju

Miller

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Activation maximization (AM) is a method used to provide interpretation for neural networks and deep neural networks. Te method observes the fundamental neurons activated by input records and identifes the particular pattern of input that maximizes the activation of the certain neuron in a certain layer [61,62]. Kaji et al [9] developed a CDSS based on recurrent neural networks (RNNs) incorporating an attention mechanism for prediction over two weeks of patients' ICU courses.…”

Section: Crc Risk To Users By Interactive Visualization Interfacementioning

confidence: 99%

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Xie

Liao

et al. 2023

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Background. Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application. Objective. This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives. Methods. A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization. Results. Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS. Conclusions. The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

show abstract

“…While the method does yield a sample in the time domain that maximizes activation for a particular class, it does not provide insight into the relative importance of different time domain features. A couple of other studies have used activation maximization for insight into the time domain ( Yoshimura et al, 2019 , 2021 ). However, they were only applied to networks trained on samples that were around 30 time points long, and EEG samples can be hundreds to thousands of time points long.…”

Section: Introductionmentioning

confidence: 99%

A Systematic Approach for Explaining Time and Frequency Features Extracted by Convolutional Neural Networks From Raw Electroencephalography Data

Ellis

Miller

Calhoun

2022

Front. Neuroinform.

View full text Add to dashboard Cite

In recent years, the use of convolutional neural networks (CNNs) for raw resting-state electroencephalography (EEG) analysis has grown increasingly common. However, relative to earlier machine learning and deep learning methods with manually extracted features, CNNs for raw EEG analysis present unique problems for explainability. As such, a growing group of methods have been developed that provide insight into the spectral features learned by CNNs. However, spectral power is not the only important form of information within EEG, and the capacity to understand the roles of specific multispectral waveforms identified by CNNs could be very helpful. In this study, we present a novel model visualization-based approach that adapts the traditional CNN architecture to increase interpretability and combines that inherent interpretability with a systematic evaluation of the model via a series of novel explainability methods. Our approach evaluates the importance of spectrally distinct first-layer clusters of filters before examining the contributions of identified waveforms and spectra to cluster importance. We evaluate our approach within the context of automated sleep stage classification and find that, for the most part, our explainability results are highly consistent with clinical guidelines. Our approach is the first to systematically evaluate both waveform and spectral feature importance in CNNs trained on resting-state EEG data.

show abstract

Toward Understanding Acceleration-based Activity Recognition Neural Networks with Activation Maximization

Cited by 8 publications

References 20 publications

A Framework for Systematically Evaluating the Representations Learned by A Deep Learning Classifier from Raw Multi-Channel Electroencephalogram Data

A Framework for Systematically Evaluating the Representations Learned by A Deep Learning Classifier from Raw Multi-Channel Electroencephalogram Data

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

A Systematic Approach for Explaining Time and Frequency Features Extracted by Convolutional Neural Networks From Raw Electroencephalography Data

Contact Info

Product

Resources

About