A deep learning algorithm for sleep stage scoring in mice based on a multimodal network with fine-tuning technique

Akada, Keishi; Yagi, Takuya; Miura, Yuji; Beuckmann, Carsten T.; Koyama, Noriyuki; Aoshima, Ken

doi:10.1016/j.neures.2021.07.003

Cited by 8 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the LightGBM model had lower agreement with human experts in scoring the REM stage (precision, 88.8%; recall, 82.2%; f1-score, 0.85). This reduced performance when scoring the REM stage was reported for previous machine learning models 2,4 . The important question is how such performance in the REM stage compares to that of human experts.…”

Section: Discussionsupporting

confidence: 67%

“…Based on these results, the overall performance of the LightGBM model for the REM stage was similar to that of human experts. Previous studies developing deep learning models have focused on evaluating the overall performance of the trained models 2,4 . A high overall performance does not guarantee usefulness in practice if the performance of the models varies largely across different recordings.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm

Wang

Kern²,

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Machine learning has been applied in recent years to categorize sleep stages (NREM, REM, and wake) using electroencephalogram (EEG) recordings; however, a well-validated sleep scoring automatic pipeline in rodent research is still not publicly available. Here, we present IntelliSleepScorer, a software package with a graphic user interface to score sleep stages automatically in mice. IntelliSleepScorer uses the light gradient boosting machine (LightGBM) to score sleep stages for each epoch of recordings. We developed LightGBM models using a large cohort of data, which consisted of 5776 h of sleep EEG and electromyogram (EMG) signals across 519 unique recordings from 124 mice. The LightGBM model achieved an overall accuracy of 95.2% and a Cohen’s kappa of 0.91, which outperforms the baseline models such as the logistic regression model (accuracy = 93.3%, kappa = 0.88) and the random forest model (accuracy = 94.3%, kappa = 0.89). The overall performance of the LightGBM model as well as the performance across different sleep stages are on par with that of the human experts. Most importantly, we validated the generalizability of the LightGBM models: (1) The LightGBM model performed well on two publicly available, independent datasets (kappa > = 0.80), which have different sampling frequency and epoch lengths; (2) The LightGBM model performed well on data recorded at a lower sampling frequency (kappa = 0.90); (3) The performance of the LightGBM model is not affected by the light/dark cycle; and (4) A modified LightGBM model performed well on data containing only one EEG and one EMG electrode (kappa > = 0.89). Taken together, the LightGBM models offer state-of-the-art performance for automatic sleep stage scoring in mice. Last, we implemented the IntelliSleepScorer software package based on the validated model to provide an out-of-box solution to sleep researchers (available for download at https://sites.broadinstitute.org/pan-lab/resources).

show abstract

Section: Discussionsupporting

confidence: 67%

Section: Discussionmentioning

confidence: 99%

IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm

Wang

Kern²,

et al. 2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…A number of studies addressed prediction outside the field of brain pathology. Task-based fMRI data have been used to predict task state (Jang et al, 2017 ; Hu et al, 2019 ; Vu et al, 2020 ; Wang et al, 2020b ; Jiang et al, 2022 ; Ngo et al, 2022 ), while EEG data were used to predict attentional state (Zhang et al, 2021 ), sleep stage (Abou Jaoude et al, 2020 ; Akada et al, 2021 ) and brain age (Levakov et al, 2020 ; Niu et al, 2020 ; Ning et al, 2021 ; Ren et al, 2022 ), recognize emotions (Wang et al, 2020a ; Ramzan and Dawn, 2021 ; Bagherzadeh et al, 2022 ; Xiao et al, 2022 ), detect P300 (Solon et al, 2019 ; Borra et al, 2021 ), cortical oscillatory activity (Abdul Nabi Ali et al, 2022 ) and cortical activity during sleep (Li et al, 2020a ). Recently, several studies have used DL to decode motor imagery (Hassanpour et al, 2019 ; Ebrahimi et al, 2020 ; Xu et al, 2020 ; Dehghani et al, 2021 ; Fan et al, 2021 ), which is important in brain-computer interface.…”

Section: Deep Learning Applications In Neuroimagingmentioning

confidence: 99%

“…In the third stage, predictions were made about MCI and AD. Akada et al ( 2021 ) found that a multimodal approach in which EEG and electromyography (EMG) data were first processed separately and then combined gave better results than a rule-based integration approach and an ensemble stacking approach.…”

Section: Challenges and Solutionsmentioning

confidence: 99%

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Avberšek

Repovš

2022

Front. Neuroimaging

View full text Add to dashboard Cite

Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.

show abstract

“…These models are then required to score new data based on what it has learned from the data with which they are familiar. [9][10][11][12][13][14][15][16] However, one concern with this approach is that networks can often perform poorly when they are asked to classify new data that they have not encountered before. 17 This could be the case with data from a different laboratory or unfamiliar experimental treatments, such as alterations in EEG/EMG produced by pharmacological agents, genetic manipulations, and/or disease models.…”

Section: Introductionmentioning

confidence: 99%

Sleep-Deep-Net learns sleep wake scoring from the end-user and completes each record in their style

Katsuki,

Spratt,

Brown

et al. 2023

Preprint

View full text Add to dashboard Cite

Sleep-wake scoring is a time-consuming, tedious but essential component of clinical and pre-clinical sleep research. Sleep scoring is even more laborious and challenging in rodents due to the smaller EEG amplitude differences between states and the rapid state transitions which necessitate scoring in shorter epochs. Although many automated rodent sleep scoring methods exist, they do not perform as well when scoring new data sets, especially those which involve changes in the EEG/EMG profile. Thus, manual scoring by expert scorers remains the gold-standard. Here we take a different approach to this problem by using a neural network to accelerate the scoring of expert scorers. Sleep-Deep-Net (SDN) creates a bespoke deep convolution neural network model for individual electroencephalographic or local-field-potential records via transfer learning of GoogleNet, by learning from a small subset of manual scores of each EEG/LFP record as provided by the end-user. SDN then automates scoring of the remainder of the EEG/LFP record. A novel REM scoring correction procedure further enhanced accuracy. SDN reliably scores EEG and LFP data and retains sleep-wake architecture in wild-type mice, in sleep induced by the hypnotic zolpidem, in a mouse model of Alzheimer’s disease and in a genetic knock-down study, when compared to manual scoring. SDN reduced manual scoring time to 1/12. Since SDN uses transfer learning on each independent recording, it is not biased by previously scored existing data sets. Thus, we find SDN performs well when used on signals altered by a drug, disease model or genetic modification.STATEMENT OF SIGNIFICANCESleep medicine is often critically advanced by translational research based onin vivoelectrophysiologic mouse data. A necessary but time-consuming step in this field is scoring epochs of recordings into wakefulness, non-rapid-eye-movement sleep and non-rapid-eye-movement sleep. Despite efforts to automate this, manual scoring remains the gold-standard since automatic methods poorly handle data that is not similar enough to data used during development. Here, we describe a novel automated sleep scoring method that involves retraining a deep-convolution-neural-net capable of computer vision to score sleep-wake patterns after learning from a small set of manual scores within a record. This avoids biasing the model to expect data to be the same as its training set from previous records.

show abstract

A deep learning algorithm for sleep stage scoring in mice based on a multimodal network with fine-tuning technique

Cited by 8 publications

References 30 publications

IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm

IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Sleep-Deep-Net learns sleep wake scoring from the end-user and completes each record in their style

Contact Info

Product

Resources

About