Symptom-Based Dispatching in an Emergency Medical Communication Centre: Sensitivity, Specificity, and the Area under the ROC Curve

Abstract

Background: Measuring the performance of emergency medical dispatch tools used in paramedic-staffed emergency medical communication centres (EMCCs) is rarely performed. The objectives of our study were, therefore, to measure the performance and accuracy of Geneva’s dispatch system based on symptom assessment, in particular, the performance of ambulance dispatching with lights and sirens (L&S) and to measure the effect of adding specific protocols for each symptom. Methods: We performed a prospective observational study including all emergency calls received at Geneva’s EMCC (Switzerland) from 1 January 2014 to 1 July 2019. The risk levels selected during the emergency calls were compared to a reference standard, based on the National Advisory Committee for Aeronautics (NACA) scale, dichotomized to severe patient condition (NACA ≥ 4) or stable patient condition (NACA < 4) in the field. The symptom-based dispatch performance was assessed using a receiver operating characteristic (ROC) curve. Contingency tables and a Fagan nomogram were used to measure the performance of the dispatch with or without L&S. Measurements were carried out by symptom, and a group of symptoms with specific protocols was compared to a group without specific protocols. Results: We found an acceptable area under the ROC curve of 0.7474, 95%CI (0.7448–0.7503) for the 148,979 assessments included in the study. Where the severity prevalence was 21%, 95%CI (20.8–21.2). The sensitivity of the L&S dispatch was 87.5%, 95%CI (87.1–87.8); and the specificity was 47.3%, 95%CI (47.0–47.6). When symptom-specific assessment protocols were used, the accuracy of the assessments was slightly improved. Conclusions: Performance measurement of Geneva’s symptom-based dispatch system using standard diagnostic test performance measurement tools was possible. The performance was found to be comparable to other emergency medical dispatch systems using the same reference standard. However, the implementation of specific assessment protocols for each symptom may improve the accuracy of symptom-based dispatch systems.

A registry-based observational study comparing emergency calls assessed by emergency medical dispatchers with and without support by registered nurses

“…Comparing the rates of predictive values between and within dispatch systems requires knowledge about the prevalence of severity levels found in the field (i.e., detected by the reference standard) [ 33 ]. Therefore, it is important to note that the prevalence of the most urgent priorities (patients assessed as RETTS Red/Orange by the ambulance) was not entirely similar among the calls assessed by each group (45% for EMD and 33% for EMD supported by RN).…”

“…International data on sensitivity for time-critical and severe calls has been reported to be as high as 93.3% and 86–87.5% for Australia and Switzerland respectively [ 33 – 35 ]. The sensitivity for cardiac arrest has been studied for different settings and reported to range from 65.9 to 86% in various countries [ 36 – 39 ].…”

Contribution of Live Video to Physicians’ Remote Assessment of Suspected COVID-19 Patients in an Emergency Medical Communication Centre: A Retrospective Study and Web-Based Survey

“…We found that callers assessed their situation fairly well themselves, as only 5 (0.1%) serious situations (for 4464 assessments) were identified following a call to the COVID-19 medical helpline, compared with 735 (16.3%) serious situations (for 4493 assessments) on the 144 number ( Table 1 ). The provision of two telephone helplines (urgent or ambulance/non-urgent) therefore contributes to the performance of the EMCC triage, as the prevalence of severity is completely different depending on which line is called [ 24 ]. The skills available and the assessment tools used in EMCCs should be adapted to the condition of the patients assessed.…”

“…They handle all calls using a computer-aided dispatch system (ICAD ® from Hexagon AB ® ). For telephone assessments of patients, Geneva’s EMDs have used the symptom-based dispatch (SBD) system derived from the Swiss Emergency Triage Scale (SETS ® ) since 2013 [ 24 ]. Historically, there was not an Emergency Physician (EP) physically present in the EMCC, although EMDs had always had remote access to a physician in case of an assessment concern or medical question.…”

Differences between the dispatch priority assessments of emergency medical dispatchers and emergency medical services: a prospective register-based study in Finland

“…The data were also dichotomised into two groups according to EMS's criteria (Table 1)-one where the situation was considered urgent and the other where the situation was considered non-urgent. This allowed comparison with other studies that used the same dichotomisation [22][23][24]. The dichotomisation was made in a specific order in which the paramedic's assessment was primarily considered (Fig.…”