Use Characteristics and Triage Acuity of a Digital Symptom Checker in a Large Integrated Health System: Population-Based Descriptive Study

Abstract

Background Pressure on the US health care system has been increasing due to a combination of aging populations, rising health care expenditures, and most recently, the COVID-19 pandemic. Responses to this pressure are hindered in part by reliance on a limited supply of highly trained health care professionals, creating a need for scalable technological solutions. Digital symptom checkers are artificial intelligence–supported software tools that use a conversational “chatbot” format to support rapid diagnosis and consistent triage. The COVID-19 pandemic has brought new attention to these tools due to the need to avoid face-to-face contact and preserve urgent care capacity. However, evidence-based deployment of these chatbots requires an understanding of user demographics and associated triage recommendations generated by a large general population. Objective In this study, we evaluate the user demographics and levels of triage acuity provided by a symptom checker chatbot deployed in partnership with a large integrated health system in the United States. Methods This population-based descriptive study included all web-based symptom assessments completed on the website and patient portal of the Sutter Health system (24 hospitals in Northern California) from April 24, 2019, to February 1, 2020. User demographics were compared to relevant US Census population data. Results A total of 26,646 symptom assessments were completed during the study period. Most assessments (17,816/26,646, 66.9%) were completed by female users. The mean user age was 34.3 years (SD 14.4 years), compared to a median age of 37.3 years of the general population. The most common initial symptom was abdominal pain (2060/26,646, 7.7%). A substantial number of assessments (12,357/26,646, 46.4%) were completed outside of typical physician office hours. Most users were advised to seek medical care on the same day (7299/26,646, 27.4%) or within 2-3 days (6301/26,646, 23.6%). Over a quarter of the assessments indicated a high degree of urgency (7723/26,646, 29.0%). Conclusions Users of the symptom checker chatbot were broadly representative of our patient population, although they skewed toward younger and female users. The triage recommendations were comparable to those of nurse-staffed telephone triage lines. Although the emergence of COVID-19 has increased the interest in remote medical assessment tools, it is important to take an evidence-based approach to their deployment.

COVID-19 in Brazil—Preliminary Analysis of Response Supported by Artificial Intelligence in Municipalities

“…In spite of all limitations, we were able to describe a successful implementation of an AI-powered chatbot to assess COVID-19 related risks and to provide optimal healthcare utilization through flexible, on-demand and accessible structures. These findings corroborate with others from Austria, Switzerland and the USA, suggesting that the use of this technology may decrease the healthcare system overload and increase access to healthcare system ( 13 – 15 ). All efforts summarized here were validated along with health authorities in the country, therefore safety was ensured based on rigorous evaluation by state departments in each city where the algorithm was implemented.…”

Quality of condition suggestions and urgency advice provided by the Ada symptom assessment app evaluated with vignettes optimised for Australia

“…Home SAA use has the potential to reduce the burden on primary and secondary care and to navigate patients to appropriate care (Winn et al 2019;Miller et al 2020;Morse et al 2020). Confirmatory prospective studies of these findings are in progress.…”

“…Confirmatory prospective studies of these findings are in progress. In a recent study, Morse et al (2020) described their experience integrating an SAA in northern California for Sutter Health, in which 26 646 assessments were delivered in a broad patient population to provide assistance outside typical physician office hours. In that study, SAA triage recommendations were comparable to those of nurse-staffed telephone triage lines.…”

“…In addition to other components of performance surveillance of the Ada app as described in Gilbert et al (2020), Miller et al (2020 and Morse et al (2020), the aim of this study was to apply the methodology and vignettes of Hill et al (2020) to provide a greater understanding of the results for the 36 free-to-Australian users SAAs evaluated by Hill et al (2020).…”

Reliability of COVID-19 symptom checkers as national triage tools: an international case comparison study

“…Some nations have gone further, using symptom checkers in place of more typical clinical triage systems. 1 …”