AI vs Humans for the diagnosis of sleep apnea

Thorey, Valentin; Hernández, Albert Bou; Arnal, Pierrick J.; During, Emmanuel

doi:10.1109/embc.2019.8856877

Cited by 16 publications

(9 citation statements)

References 14 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AASM (American Academy of Sleep Medicine) guidelines define five different sleep stages: wake, NREM1, NREM2, NREM3 (or deep sleep), and REM (rapid eye movement) [1]. Sleep stage classification is a primary tool used systematically in the diagnosis of sleep disorders such as narcolepsy [2] or sleep apnea [3]. of the five sleep stages according to the standards defined by the AASM.…”

Section: Introductionmentioning

confidence: 99%

RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale

Guillot¹,

Thorey²

2021

IEEE Trans. Neural Syst. Rehabil. Eng.

Self Cite

View full text Add to dashboard Cite

Sleep disorder diagnosis relies on the analysis of polysomnography (PSG) records. As a preliminary step of this examination, sleep stages are systematically determined. In practice, sleep stage classification relies on the visual inspection of 30second epochs of polysomnography signals. Numerous automatic approaches have been developed to replace this tedious and expensive task. Although these methods demonstrated better performance than human sleep experts on specific datasets, they remain largely unused in sleep clinics. The main reason is that each sleep clinic uses a specific PSG montage that most automatic approaches cannot handle out-of-the-box. Moreover, even when the PSG montage is compatible, publications have shown that automatic approaches perform poorly on unseen data with different demographics. To address these issues, we introduce RobustSleepNet, a deep learning model for automatic sleep stage classification able to handle arbitrary PSG montages. We trained and evaluated this model in a leave-one-out-dataset fashion on a large corpus of 8 heterogeneous sleep staging datasets to make it robust to demographic changes. When evaluated on an unseen dataset, RobustSleepNet reaches 97% of the F1 of a model explicitly trained on this dataset. Hence, RobustSleepNet unlocks the possibility to perform high-quality out-of-the-box automatic sleep staging with any clinical setup. We further show that finetuning RobustSleepNet, using a part of the unseen dataset, increases the F1 by 2% when compared to a model trained specifically for this dataset. Therefore, finetuning might be used to reach a state-of-the-art level of performance on a specific population.

show abstract

Section: Introductionmentioning

confidence: 99%

RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale

Guillot¹,

Thorey²

2021

IEEE Trans. Neural Syst. Rehabil. Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The values from Table 5 can be compared with inter-scorer variability. In [ 41 ] mean absolute error for AHI calculation was presented for five different scorers. These errors range from 3.82 to 5.15.…”

Section: Discussionmentioning

confidence: 99%

An LSTM Network for Apnea and Hypopnea Episodes Detection in Respiratory Signals

Drzazga

Cyganek

2021

Sensors

View full text Add to dashboard Cite

One of the most common sleep disorders is sleep apnea. It manifests itself by episodes of shallow breathing or pauses in breathing during the night. Diagnosis of this disease involves polysomnography examination, which is expensive. Alternatively, diagnostic doctors can be supported with recordings from the in-home polygraphy sensors. Furthermore, numerous attempts for providing an automated apnea episodes annotation algorithm have been made. Most of them, however, do not distinguish between apnea and hypopnea episodes. In this work, a novel solution for epoch-based annotation problem is presented. Utilizing an architecture based on the long short-term memory (LSTM) networks, the proposed model provides locations of sleep disordered breathing episodes and identifies them as either apnea or hypopnea. To achieve this, special pre- and postprocessing steps have been designed. The obtained labels can be then used for calculation of the respiratory event index (REI), which serves as a disease severity indicator. The input for the model consists of the oronasal airflow along with the thoracic and abdominal respiratory effort signals. Performance of the proposed architecture was verified on the SHHS-1 and PhysioNet Sleep databases, obtaining mean REI classification error of 9.24/10.52 with standard deviation of 11.61/7.92 (SHHS-1/PhysioNet). Normal breathing, hypopnea and apnea differentiation accuracy is assessed on both databases, resulting in the correctly classified samples percentage of 86.42%/84.35%, 49.30%/58.28% and 68.20%/69.50% for normal breathing, hypopnea and apnea classes, respectively. Overall accuracies are 80.66%/82.04%. Additionally, the effect of wake periods is investigated. The results show that the proposed model can be successfully used for both episode classification and REI estimation tasks.

show abstract

“…However, one recent study compared the event-by-event detection performance against a concensus score of five technicians. They reported an average human performance quantified by F1 of 0.55, and an F1 score from the automatic method of 0.57 [54] Similarly, Nassi et al recently proposed their WaveNet model for precisely annotating SDB events in 1 s bins. Although their model also included post-processing of the bins, they obtained a mean F1 score across events of 0.406.…”

Section: Comparison With State-of-the-art Multi-event Detectionmentioning

confidence: 99%

MSED: A Multi-Modal Sleep Event Detection Model for Clinical Sleep Analysis

Olesen

Jennum

Mignot

et al. 2023

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

Clinical sleep analysis require manual analysis of sleep patterns for correct diagnosis of sleep disorders. However, several studies have shown significant variability in manual scoring of clinically relevant discrete sleep events, such as arousals, leg movements, and sleep disordered breathing (apneas and hypopneas). We investigated whether an automatic method could be used for event detection and if a model trained on all events (joint model) performed better than corresponding event-specific models (single-event models). We trained a deep neural network event detection model on 1653 individual recordings and tested the optimized model on 1000 separate hold-out recordings. F1 scores for the optimized joint detection model were 0.70, 0.63, and 0.62 for arousals, leg movements, and sleep disordered breathing, respectively, compared to 0.65, 0.61, and 0.60 for the optimized single-event models. Index values computed from detected events correlated positively with manual annotations (r 2 = 0.73, r 2 = 0.77, r 2 = 0.78, respectively). We furthermore quantified model accuracy based on temporal difference metrics, which improved overall by using the joint model compared to single-event models. Our automatic model jointly detects arousals, leg movements and sleep disordered breathing events with high correlation with human annotations. Finally, we benchmark against previous state-of-the-art multi-event detection models and found an overall increase in F1 score with our proposed model despite a 97.5% reduction in model size. Source code for training and inference is available at https://github.com/neergaard/msed.git.

show abstract

AI vs Humans for the diagnosis of sleep apnea

Cited by 16 publications

References 14 publications

RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale

RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale

An LSTM Network for Apnea and Hypopnea Episodes Detection in Respiratory Signals

MSED: A Multi-Modal Sleep Event Detection Model for Clinical Sleep Analysis

Contact Info

Product

Resources

About