Triage accuracy and the safety of user-initiated symptom assessment with an electronic symptom checker (ESC) in a real-life setting (Preprint)

Abstract

BACKGROUND Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. Variation exists between different ESCs and the conditions being assessed, including the triage capabilities. There is a limited amount of evidence on live clinical patient user safety hazards associated with the use of ESCs, as safety has mainly been evaluated with the use of clinical vignettes. National Omaolo Digital services (Omaolo) in Finland consist of an ESC for various symptoms. Users can triage their symptoms with the help of ESC algorithms. As a result of completing the ESC, the user receives a recommendation for action and a service assessment with appropriate guidance regarding their health problems for a specific symptom. This allows users to be provided with healthcare services regardless of time and place. OBJECTIVE This study investigates how well triage by the ESC matches the triage of nurses among the chief symptom assessments available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, urinary tract infection). In addition, this study assesses the safety of the Omaolo ESC. METHODS This is a clinical validation study in a real-life setting. It is a multicenter study in primary health centers across Finland. The included units were of the walk-in model of primary care, where no prior phone call or online contact is required. Upon arriving at the health center, users (patients) filled out the ESC questions and after that a nurse assessed their triage. Findings concerning 877 patients were analyzed. The triage recommendations by the ESC were matched with the triage assessments by nurses. A case was defined as safe if the recommendation for action given by the ESC was at most one degree of urgency less urgent than the triage assessment by the nurse concerning the same case. Critical cases were analyzed in detail. RESULTS The mean of the exact match for all symptom assessments was 53.7%. The mean of the exact match or overly conservative but suitable for all symptom assessments was 67.7%. Safe assessments by the ESC accounted for 97.6% of all assessments made. Critical cases were identified and further analyzed. In these 21 cases, no indication was found that patient safety had been compromised. CONCLUSIONS This study was the first to assess the accuracy and safety of the Omaolo ESC in the Finnish primary health care context. The findings indicate that the Omaolo ESC is safe compared to the assessment and triage of a triage nurse. INTERNATIONAL REGISTERED REPORT RR2-10.2196/41423