Patients’ Utilization and Perception of an Artificial Intelligence–Based Symptom Assessment and Advice Technology in a British Primary Care Waiting Room: Exploratory Pilot Study

Miller, Stephen J.; Gilbert, Stephen; Virani, Vishaal; Wicks, Paul

doi:10.2196/19713

Cited by 92 publications

(96 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Research supported by Ada Health GmbH shows that Ada had the highest top 3 suggestion diagnostic accuracy (70.5%) compared to other SCs [ 19 ], and the correct condition was among the first three results in 83% in an Australian assessment study [ 20 ]. Similarly to our results, the majority of patients would recommend Ada (85.3%) to friends or relatives [ 21 ].…”

Section: Discussionsupporting

confidence: 89%

Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

et al. 2021

View full text Add to dashboard Cite

Background Timely diagnosis and treatment are essential in the effective management of inflammatory rheumatic diseases (IRDs). Symptom checkers (SCs) promise to accelerate diagnosis, reduce misdiagnoses, and guide patients more effectively through the health care system. Although SCs are increasingly used, there exists little supporting evidence. Objective To assess the diagnostic accuracy, patient-perceived usability, and acceptance of two SCs: (1) Ada and (2) Rheport. Methods Patients newly presenting to a German secondary rheumatology outpatient clinic were randomly assigned in a 1:1 ratio to complete Ada or Rheport and consecutively the respective other SCs in a prospective non-blinded controlled randomized crossover trial. The primary outcome was the accuracy of the SCs regarding the diagnosis of an IRD compared to the physicians’ diagnosis as the gold standard. The secondary outcomes were patient-perceived usability, acceptance, and time to complete the SC. Results In this interim analysis, the first 164 patients who completed the study were analyzed. 32.9% (54/164) of the study subjects were diagnosed with an IRD. Rheport showed a sensitivity of 53.7% and a specificity of 51.8% for IRDs. Ada’s top 1 (D1) and top 5 disease suggestions (D5) showed a sensitivity of 42.6% and 53.7% and a specificity of 63.6% and 54.5% concerning IRDs, respectively. The correct diagnosis of the IRD patients was within the Ada D1 and D5 suggestions in 16.7% (9/54) and 25.9% (14/54), respectively. The median System Usability Scale (SUS) score of Ada and Rheport was 75.0/100 and 77.5/100, respectively. The median completion time for both Ada and Rheport was 7.0 and 8.5 min, respectively. Sixty-four percent and 67.1% would recommend using Ada and Rheport to friends and other patients, respectively. Conclusions While SCs are well accepted among patients, their diagnostic accuracy is limited to date. Trial registration DRKS.de, DRKS00017642. Registered on 23 July 2019

show abstract

Section: Discussionsupporting

confidence: 89%

Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Most of the patients in that study (274/304, 90.1%) reported receiving useful information for their health problems and reported that they would use the symptom checker again (278/304, 91.4%) [ 14 ]. These findings are in line with another study that examined perspectives on use of the “Ada” symptom checker, which showed that most of the participants (443/503, 88.1%) would recommend the platform to a relative [ 16 ]. An important factor that seems to influence symptom checker acceptance is age, with younger populations exhibiting higher acceptance [ 14 , 15 ].…”

Section: Introductionsupporting

confidence: 88%

Young Adults’ Perspectives on the Use of Symptom Checkers for Self-Triage and Self-Diagnosis: Qualitative Study

Aboueid¹,

Meyer²,

Wallace³

et al. 2021

JMIR Public Health Surveill

View full text Add to dashboard Cite

Background Young adults often browse the internet for self-triage and diagnosis. More sophisticated digital platforms such as symptom checkers have recently become pervasive; however, little is known about their use. Objective The aim of this study was to understand young adults’ (18-34 years old) perspectives on the use of the Google search engine versus a symptom checker, as well as to identify the barriers and enablers for using a symptom checker for self-triage and self-diagnosis. Methods A qualitative descriptive case study research design was used. Semistructured interviews were conducted with 24 young adults enrolled in a university in Ontario, Canada. All participants were given a clinical vignette and were asked to use a symptom checker (WebMD Symptom Checker or Babylon Health) while thinking out loud, and were asked questions regarding their experience. Interviews were audio-recorded, transcribed, and imported into the NVivo software program. Inductive thematic analysis was conducted independently by two researchers. Results Using the Google search engine was perceived to be faster and more customizable (ie, ability to enter symptoms freely in the search engine) than a symptom checker; however, a symptom checker was perceived to be useful for a more personalized assessment. After having used a symptom checker, most of the participants believed that the platform needed improvement in the areas of accuracy, security and privacy, and medical jargon used. Given these limitations, most participants believed that symptom checkers could be more useful for self-triage than for self-diagnosis. Interestingly, more than half of the participants were not aware of symptom checkers prior to this study and most believed that this lack of awareness about the existence of symptom checkers hindered their use. Conclusions Awareness related to the existence of symptom checkers and their integration into the health care system are required to maximize benefits related to these platforms. Addressing the barriers identified in this study is likely to increase the acceptance and use of symptom checkers by young adults.

show abstract

“…However, handover to HCPs was not carried out in their study, nor did the tool have a system to enable handover. A number of studies 9,20,21 have reported apps for self-assessment of urgency/triage in the general practice and acute primary care setting, but evaluation of the handover of history and symptom information to HCPs was not addressed.…”

Section: Discussionmentioning

confidence: 99%

“…A previous study found that patients using Ada's tool in a primary care waiting room found it helpful and easy to use. 9 If appropriately adapted to the setting, the benefits of a digital ED history taking tool could assist nurse-led triage in the waiting room and/or assessment and treatment by ED physicians.…”

Section: Introductionmentioning

confidence: 99%

Improving emergency department patient-doctor conversation through an artificial intelligence symptom taking tool: an action-oriented design pilot study

Scheder-Bieschin

Bluemke²,

Buijzer³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

IMPORTANCECommunication between patients and healthcare professionals is frequently challenging in the crowded emergency department (ED), with few opportunities to develop rapport or empathy. Digital tools for patients and physicians have been proposed as helpful but their utility is not established.OBJECTIVETo evaluate a patient-facing digital symptom and history taking, as well as handover tool in the waiting room.DESIGNA two-phase, questionnaire-based quality improvement study. Phase I observations guided iterative improvement, which was then further evaluated in Phase II.SETTINGED of a German tertiary referral and major trauma hospital providing interdisciplinary treatment for an average of 120 patients daily.PARTICIPANTSAll patients who were willing/able to provide consent, excluding patients: (i) with severe injury/illness requiring immediate treatment; (ii) with traumatic injury; (iii) incapable of completing a health assessment; or, (iv) under 18 years old. Of 1699 patients presenting to the ED, 815 were eligible based on triage level. With available recruitment staff, 135 were approached, of whom 81 were included in the study.INTERVENTION/OBSERVATIONPatients entered information into the tool, which generated a handover report to be accessed via a clinician dashboard. All users completed evaluation questionnaires. Clinicians were trained to observationally assess the tool as a prototype, without relying upon it for clinical care.MAIN OUTCOMES AND MEASURESPatient and clinician Likert scale ratings of tool performance.RESULTSRespondents were strongly positive in endorsing the tool’s usefulness in facilitating conversation (75% of patients, 73% physicians, 100% nurses). Nurses judged the tool as potentially time saving, whilst physicians assessed it as time saving only in some ED medical specialisms (e.g. Surgery). Patients understood the tool questions and reported high usability. The proportion of patients, physicians and nurses who would recommend the tool was 78%, 53% and 76%.CONCLUSIONS AND RELEVANCEThe system has clear potential to improve patient-HCP interaction and make efficiency savings in the ED. Future research and development will extend the range of patients for which the history collection has clinical utility.Key PointsQuestionCan a patient-facing digital symptom and clinical history taking tool provide conversational support, aid in symptom taking, facilitate record keeping, and lead to improved rapport between patients, physicians and nurses in the emergency department (ED)?FindingsAcceptability was high, with improved rapport experienced 90% of the time for patients, 73% for physicians and 100% for nurses. Nurses assessed the tool as having workflow benefit through potential time saving. Physicians assessed the current tool design as providing time saving in certain ED medical specialisms including Surgery.MeaningThe patient-facing tool for symptom and history taking provided meaningful conversation support and showed potential for efficiency savings, however, further research and testing is required before time savings can be consistently delivered to ED clinicians across the range of relevant ED medical specialisms.

show abstract

Patients’ Utilization and Perception of an Artificial Intelligence–Based Symptom Assessment and Advice Technology in a British Primary Care Waiting Room: Exploratory Pilot Study

Cited by 92 publications

References 26 publications

Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

Young Adults’ Perspectives on the Use of Symptom Checkers for Self-Triage and Self-Diagnosis: Qualitative Study

Improving emergency department patient-doctor conversation through an artificial intelligence symptom taking tool: an action-oriented design pilot study

Contact Info

Product

Resources

About