Validation of the National Emergency Department Overcrowding Score (NEDOCS) in a UK non-specialist emergency department

Abstract

IntroductionEmergency department (ED) crowding has significant adverse consequences, however, there is no widely accepted tool to measure it. This study validated the National Emergency Department Overcrowding score (NEDOCS) (range 0–200 points), which uses routinely collected ED data.MethodsThis prospective single-centre study sampled data during four periods of 2018. The outcome against which NEDOCS performance was assessed was a composite of clinician opinion of crowding (physician and nurse in charge). Area under thereceiver operating characteristic curves (AUROCs) and calibration plots were produced. Six-hour stratified sampling was added to adjust for temporal correlation of clinician opinion. Staff inter-rater agreement and NEDOCS association with opinion of risk, safety and staffing levels were collected.ResultsFrom 905 sampled hours, 448 paired observations were obtained, with the ED deemed crowded 18.5% of the time. Inter-rater agreement between staff was moderate (weighted kappa 0.57 (95% CI 0.56 to 0.60)). AUROC for NEDOCS was 0.81 (95% CI 0.77 to 0.86). Adjusted for temporal correlation, AUROC was 0.80 (95% CI 0.73 to 0.88). At a cut-off of 100 points sensitivity was 75.9% (95% CI 65.3% to 84.6%), specificity 72.1% (95% CI 67.1% to 76.6%), positive predictive value 38.2% (95% CI 30.7% to 46.1%) and negative predictive value 92.9% (95% CI 89.3% to 95.6%). NEDOCS underpredicted clinical opinion on Calibration assessment, only partially correcting with intercept updating. For perceived risk of harm, safety and insufficient staffing, NEDOCS AUROCs were 0.71 (95% CI 0.61 to 0.82), 0.71 (95% CI 0.63 to 0.80) and 0.70 (95% CI 0.64 to 0.76), respectively.ConclusionsNEDOCS demonstrated good discriminatory power for clinical perception of crowding. Prior to implementation, determining individual unit ED cut-off point(s) would be important as published thresholds may not be generalisable. Future studies could explore refinement of existing variables or addition of new variables, including acute physiological data, which may improve performance.

Comparison of the International Crowding Measure in Emergency Departments (ICMED) and the National Emergency Department Overcrowding Study (NEDOCS) in Tertiary Care Hospital to Measure Emergency Department Crowding

“…Whether the measure works well outside the USA. 17,18 A study conducted by Raj et al in Australia concluded that the NEDOCS tool needed to be more valid in their setting, owing to inconsistencies in the reflection of the staff's sense of 'overcrowding'. However, they opined that NEDOCS is a significant instrument which has the potential to be improved.…”

Differentiating properties of occupancy rate and workload to estimate crowding: A Swedish national cross‐sectional study

“… 14 , 17 , 18 A similar methodology has been used for the National Emergency Department Overcrowding Study with comparable, replicable results. 19 , 20 , 21 Mixed model linear regression, using the restricted maximum likelihood function, was used to test the effects of TTB on OR with additional fixed effects and hospital type, hospital size, staff‐to‐patient ratio, and ED as random effect variables. The effects of OR and TTB on assessed workload were investigated similarly.…”

Predicting Ambulance Patient Wait Times: A Multicenter Derivation and Validation Study

“…There is no literature regarding which predictor variables or techniques should be used, nor is there information about whether a single site-derived model would apply to multiple emergency departments for systemwide implementation. Cooney et al 5,6 considered whether the National Emergency Department Overcrowding Score (NEDOCS) 7,8 may be a useful predictor variable or model, with the association describing the relationship between access block and ambulance ramping. Others have explored centralized ambulance patient distribution optimization.…”