Several approaches exist today for developing predictive models across multiple clinical sites, yet there is a lack of comparative data on their performance, especially within the context of EHR-based prediction models. We set out to provide a framework for prediction across healthcare settings. As a case study, we examined an ED disposition prediction model across three geographically and demographically diverse sites. We conducted a 1-year retrospective study, including all visits in which the outcome was either discharge-to-home or hospitalization. Four modeling approaches were compared: a ready-made model trained at one site and validated at other sites, a centralized uniform model incorporating data from all sites, multiple site-specific models, and a hybrid approach of a ready-made model re-calibrated using site-specific data. Predictions were performed using XGBoost. The study included 288,962 visits with an overall admission rate of 16.8% (7.9–26.9%). Some risk factors for admission were prominent across all sites (e.g., high-acuity triage emergency severity index score, high prior admissions rate), while others were prominent at only some sites (multiple lab tests ordered at the pediatric sites, early use of ECG at the adult site). The XGBoost model achieved its best performance using the uniform and site-specific approaches (AUC = 0.9–0.93), followed by the calibrated-model approach (AUC = 0.87–0.92), and the ready-made approach (AUC = 0.62–0.85). Our results show that site-specific customization is a key driver of predictive model performance.

No abstract

Background: Academic emergency medicine is a constant balance between efficiency and education. We developed a new model called swarming, where the bedside nurse, resident, and attending/fellow simultaneously evaluate the patient, including initial vital signs, bedside triage, focused history and physical examination, and discussion of the treatment plan, thus creating a shared mental model. Objectives:To combine perceptions from trainee physicians, supervising physicians, nurses, and families with in vivo measurements of emergency department swarms to better conceptualize the swarming model.Methods: This mixed methods study was conducted using a convergent design. Qualitative data from focus groups with nurses, residents, and attendings/fellows were analyzed using directed content analysis. Swarming encounters were observed in real time; durations of key aspects and family satisfaction scores were analyzed using descriptive statistics. The qualitative and quantitative findings were integrated a posteriori.Results: From the focus group data, 54 unique codes were identified, which were grouped together into five larger themes. From 39 swarms, mean (AESD) time (minutes) spent in patient rooms: nurses = 6.8 (AE3.0), residents = 10.4 (AE4.1), and attendings/fellows = 9.4 (AE4.3). Electronic documentation was included in 67% of swarms, and 39% included orders initiated at the bedside. Mean (AESD) family satisfaction was 4.8 (AE0.7; Likert scale 1-5). Conclusions:Swarming is currently implemented with significant variability but results in high provider and family satisfaction. There is also consensus among physicians that swarming improves trainee education in the emergency setting. The benefits and barriers to swarming are underscored by the unpredictable nature of the ED and the observed variability in implementation. Our findings provide a critical foundation for our efforts to refine, standardize, and appraise our swarming model.