Interobserver Agreement Among Sleep Scorers From Different Centers in a Large Dataset

Norman, Robert G.; Pal, Ivan; Stewart, Chip; Walsleben, Joyce A.; Rapoport, David M.

doi:10.1093/sleep/23.7.1e

Cited by 203 publications

(195 citation statements)

References 0 publications

Supporting

Mentioning

160

Contrasting

Unclassified

Order By: Relevance

“…6 The improvement in scoring agreement is more impressive when compared to historical values for R&K scoring, such as the 73% agreement among 5 experts from different centers reported by Norman and colleagues. 7 Our data, like those of other researchers, indicates that the best agreement is achieved with stages W, N2, and R. Disagreement with the scoring of stage N1 includes scoring of stage W and scoring of stage N2 sleep. Disagreement with the scoring of stage N3 sleep is almost entirely based on confusion with scoring stage N2 sleep.…”

Section: Discussionsupporting

confidence: 85%

The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage Scoring

Rosenberg

Hout²

2013

Journal of Clinical Sleep Medicine

344

228

View full text Add to dashboard Cite

Grigg-Damberger, in a recent review, 3 noted that there have been few complaints about the new scoring system. Moser and colleagues 4 found signifi cant differences in stage scores with the AASM Manual. Stages N1 and N3 were increased, whereas stage N2 was decreased with the new scoring as compared to the old. Scorers were drawn from a pool of 30 experienced sleep experts. Danker-Hopfe and colleagues found that use of the AASM Manual resulted in slightly improved inter-rater reliability to 82.0% compared to 80.6% using R & K rules. 5 Her study was based on 72 recordings that were scored with both methods by a pool of 7 experienced scorers.The AASM inter-scorer reliability (ISR) program was developed to aid sleep centers in fulfi lling accreditation standards. The standards require that a sample of randomly chosen records be scored by the center director and each of the technologists involved in record scoring. As a means of achieving this standard, the AASM ISR program provides a record each

show abstract

Section: Discussionsupporting

confidence: 85%

The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage Scoring

Rosenberg

Hout²

2013

Journal of Clinical Sleep Medicine

344

228

View full text Add to dashboard Cite

show abstract

“…However, the Cohen κ coefficients for the various stages ranged from 0.46-0.59, which are considered as moderate agreement according to Cohen κ criteria. 28 Moreover, these results are essentially within the variability range reported in some studies comparing registered PSG scorers, 29,30 and similar to the agreement reported in comparisons between automated PSG scoring and manual scoring. 31 Multiple scoring sites, various recording techniques and a mixture between patients and normal subjects were used in our study, which may have increased biological and methodological variability compared to prior single center studies.…”

Section: Light/deep Sleep Detectionsupporting

confidence: 88%

Sleep Staging Based on Autonomic Signals: A Multi-Center Validation Study

Hedner

White

Malhotra

et al. 2011

Journal of Clinical Sleep Medicine

123

View full text Add to dashboard Cite

Study Objectives: One of the most important caveats of ambulatory devices is the inability to record and stage sleep. We assessed an algorithm determining 4 different stages: wake, light sleep, deep sleep, and REM sleep using signals derived from the portable monitor Watch-PAT100 (PAT recorder). Methods: Participants (38 normal subjects and 189 patients with obstructive sleep apnea [OSA]) underwent simultaneous overnight recordings with polysomnography (PSG) and the PAT recorder in a study originally designed to assess the accuracy of the PAT recorder in diagnosing OSA. Light/deep sleep and REM sleep from the PAT recorder recording were automatically scored based on features extracted from time series of peripheral arterial tone amplitudes and inter pulse periods. The PSG scored sleep stages 1 and 2 were classified as light sleep for epoch-by-epoch comparisons. Results:The overall agreement in detecting light/deep and REM sleep were 88.6% ± 5.9% and 88.7% ± 5.5%, respectively. There was a good agreement between PSG and the PAT recorder in quantifying sleep efficiency (78.4% ± 9.9% vs. 78.8% ± 13.4%), REM latency (237 ± 148 vs. 225 ± 159 epochs), and REM percentage (14.4% ± 6.5% vs. 19.3% ± 8.7%). OSA severity did not affect the sensitivity and specificity of the algorithm. The Cohen κ coefficient for detecting all sleep stages: sleep from wake, REM from NREM sleep, and deep from light sleep were 0.48, 0.55, 0.59, and 0.46, respectively. Conclusions: Analysis of autonomic signals from the PAT recorder can detect sleep stages with moderate agreement to more standard techniques in normal subjects and OSA patients. This novel algorithm may provide insights on sleep and sleep architecture when applying the PAT recorder for OSA diagnosis.

show abstract

“…However, we do not have a suitable measure of how well the different micro-sleep events are captured. In the case of the control subjects, the manual scores might be a suitable gold standard, although the mean inter-rater agreement rate has been reported to be as low as 76% (Norman et al, 2000). In the patients' case, the inter-rater agreement rates are lower, and in particularly in patients with PD, where the agreement rate has been reported to be only ''fair'' in more than 25% of sleep recordings (DankerHopfe et al, 2004).…”

Section: Discussionmentioning

confidence: 99%

Sleep stability and transitions in patients with idiopathic REM sleep behavior disorder and patients with Parkinson’s disease

Christensen

Jennum

Koch

et al. 2016

Clinical Neurophysiology

View full text Add to dashboard Cite

Interobserver Agreement Among Sleep Scorers From Different Centers in a Large Dataset

Cited by 203 publications

References 0 publications

The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage Scoring

The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage Scoring

Sleep Staging Based on Autonomic Signals: A Multi-Center Validation Study

Sleep stability and transitions in patients with idiopathic REM sleep behavior disorder and patients with Parkinson’s disease

Contact Info

Product

Resources

About