Agreement and prognostic accuracy of three ED vulnerability screeners: findings from a prospective multi-site cohort study

Abstract

Objectives To evaluate the agreement between three emergency department (ED) vulnerability screeners, including the InterRAI ED Screener, ER2, and PRISMA-7. Our secondary objective was to evaluate the discriminative accuracy of screeners in predicting discharge home and extended ED lengths-of-stay (> 24 h). Methods We conducted a nested sub-group study using data from a prospective multi-site cohort study evaluating frailty in older ED patients presenting to four Quebec hospitals. Research nurses assessed patients consecutively with the three screeners. We employed Cohen's Kappa to determine agreement, with high-risk cut-offs of three and four for the PRISMA-7, six for the ER2, and five for the interRAI ED Screener. We used logistic regression to evaluate the discriminative accuracy of instruments, testing them in their dichotomous, full, and adjusted forms (adjusting for age, sex, and hospital academic status). Results We evaluated 1855 older ED patients across the four hospital sites. The mean age of our sample was 84 years. Agreement between the interRAI ED Screener and the ER2 was fair (K = 0.37; 95% CI 0.33–0.40); agreement between the PRISMA-7 and ER2 was also fair (K = 0.39; 95% CI = 0.36–0.43). Agreement between interRAI ED Screener and PRISMA-7 was poor (K = 0.19; 95% CI 0.16–0.22). Using a cut-off of four for PRISMA-7 improved agreement with the ER2 (K = 0.55; 95% CI 0.51–0.59) and the ED Screener (K = 0.32; 95% CI 0.2–0.36). When predicting discharge home, the concordance statistics among models were similar in their dichotomous (c = 0.57–0.61), full (c = 0.61–0.64), and adjusted forms (c = 0.63–0.65), and poor for all models when predicting extended length-of-stay. Conclusion ED vulnerability scores from the three instruments had a fair agreement and were associated with important patient outcomes. The interRAI ED Screener best identifies older ED patients at greatest risk, while the PRISMA-7 and ER2 are more sensitive instruments.