REST: Robust and Efficient Neural Networks for Sleep Monitoring in the Wild

Duggal, Rahul; Freitas, Scott; Xiao, Cao; Chau, Duen Horng; Sun, Jimeng

doi:10.1145/3366423.3380241

Cited by 15 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, architectural changes have also resulted in reduced energy consumption. Models like MobileNet [14] and REST [9] use techniques like compression and early exiting to reduce parameter counts and, therefore, energy consumption. Suggesting alternatives and estimating updated energy consumption based on these techniques might provide users with another avenue to reduce energy usage.…”

Section: Ongoing Work and Conclusionmentioning

confidence: 99%

EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models

Shaikh

Saad-Falcon

Wright

et al. 2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

Figure 1: The EnergyVis user interface, with multiple coordinated views. (A) The Model Energy Profile View allows users to select an energy profile of pre-loaded models, generate new profiles (for models that a user wishes to train), and import saved profiles. (B) The Consumption Chart allows users to view the energy and carbon consumption of their selected model. (C) Using the Model Region view, users can view the region where a model was trained, and select regions with a lower energy intensity as an alternative to reduce emissions. (D) Users can expand the Colored Equations for succinct descriptions of various variables and how they contribute to calculating a model's emissions. (E) Finally, users can view or adjust hardware used to train a model using Alternative Hardware.

show abstract

Section: Ongoing Work and Conclusionmentioning

confidence: 99%

EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models

Shaikh

Saad-Falcon

Wright

et al. 2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Olah et al [40] highlights many examples of similar neurons in InceptionV1 and visualizes which concepts are detected by such neurons; however, the examples are manually curated by the authors. Identifying neurons that discover similar concepts also has practical benefits: in the neural network compression community, several methods [14,15,21,25,56] leverage potential neuron redundancies to generate compressed models while maintaining prediction accuracy. Even though these methods can measure neurons' similarity, there is limited work in interpreting their semantic similarity.…”

Section: Semantic Similarity Of Neuronsmentioning

confidence: 99%

“…2 shows that the "dog face" concept is detected by multiple neurons in InceptionV1 model. Although it is a well-documented phenomenon that multiple neurons detect similar features [40] (especially in model pruning research [14,21,25,56]), there is a lack of research in (1) developing scalable summarization techniques to discover concepts collectively learned by multiple neurons, and (2) enabling users to interactively interpret such concepts and their similarities. NEURO-CARTOGRAPHY aims to fill this critical research gap.…”

Section: Introductionmentioning

confidence: 99%

NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks

Park¹,

Das²,

Duggal³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Clicking a neuron adds it to the Neuron Neighbor View. Neuron Projection View1. Sarah starts exploring graph view. She selects neuron cluster for "dog face'' (pink). Graph ViewCluster Popup 2. Sarah explores neuron embedding, and wonders why one "dog face" neuron is farther away from the rest.3. Sarah discovers the "dog face" neuron is surrounded by other dog-related concepts like "furry body" and "furry head".

show abstract

“…More notably, two different configurations (M-11, M-18) were proposed based on the depth of the network. Moreover, we experimented on the SORS architecture, which was specifically proposed for sleep staging [13] and was adopted as a baseline in a previous work [23]. We adopted hyperparameter settings identical to those used in the original works.…”

Section: Model Architecturementioning

confidence: 99%

“…The Sleep-EDF dataset [24,25] consists of 39 EEG recordings from healthy subjects. These recordings have been adopted as a baseline dataset for several sleep staging studies [5,6,23]. In addition to the Sleep-EDF dataset, we further evaluated our method on the Sleep-ISRUC dataset [26].…”

Section: Datasetmentioning

confidence: 99%

Training Neural Networks with Domain Pattern-Aware Auxiliary Task for Sleep Staging

Lee¹,

Hwang²,

Lee³

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Recent studies on deep learning for biomedical data analysis have shown that guiding neural networks (NNs) to incorporate appropriate domain knowledge can enhance model performance. Accordingly, we present an auxiliary classification task for sleep staging to enable NNs to exploit clinically significant EEG patterns in data. Specifically, representations constructed from a global max pooling operator, promoting the NNs to extract distinctive signal patterns for each sleep stage, are adopted herein. While previous studies required additional training processes or parameter tuning phases for correct formulation of the domain knowledge during training, our method operates in an end-to-end manner without further calibration. During the experiments, the proposed method improved the classification accuracies compared to the original model. Moreover, intermediate representations in the proposed method were highly interpretable, wherein medically-significant features were correctly addressed with the activation values. Overall, the proposed method successfully augmented model qualities in two important aspects for adoption of AI in medical tasks: good model performance and clinically sound interpretation of models.

show abstract

REST: Robust and Efficient Neural Networks for Sleep Monitoring in the Wild

Cited by 15 publications

References 17 publications

EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models

EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models

NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks

Training Neural Networks with Domain Pattern-Aware Auxiliary Task for Sleep Staging

Contact Info

Product

Resources

About