Determinants of Laypersons’ Trust in Medical Decision Aids: Randomized Controlled Trial

Abstract

Background Symptom checker apps are patient-facing decision support systems aimed at providing advice to laypersons on whether, where, and how to seek health care (disposition advice). Such advice can improve laypersons’ self-assessment and ultimately improve medical outcomes. Past research has mainly focused on the accuracy of symptom checker apps’ suggestions. To support decision-making, such apps need to provide not only accurate but also trustworthy advice. To date, only few studies have addressed the question of the extent to which laypersons trust symptom checker app advice or the factors that moderate their trust. Studies on general decision support systems have shown that framing automated systems (anthropomorphic or emphasizing expertise), for example, by using icons symbolizing artificial intelligence (AI), affects users’ trust. Objective This study aims to identify the factors influencing laypersons’ trust in the advice provided by symptom checker apps. Primarily, we investigated whether designs using anthropomorphic framing or framing the app as an AI increases users’ trust compared with no such framing. Methods Through a web-based survey, we recruited 494 US residents with no professional medical training. The participants had to first appraise the urgency of a fictitious patient description (case vignette). Subsequently, a decision aid (mock symptom checker app) provided disposition advice contradicting the participants’ appraisal, and they had to subsequently reappraise the vignette. Participants were randomized into 3 groups: 2 experimental groups using visual framing (anthropomorphic, 160/494, 32.4%, vs AI, 161/494, 32.6%) and a neutral group without such framing (173/494, 35%). Results Most participants (384/494, 77.7%) followed the decision aid’s advice, regardless of its urgency level. Neither anthropomorphic framing (odds ratio 1.120, 95% CI 0.664-1.897) nor framing as AI (odds ratio 0.942, 95% CI 0.565-1.570) increased behavioral or subjective trust (P=.99) compared with the no-frame condition. Even participants who were extremely certain in their own decisions (ie, 100% certain) commonly changed it in favor of the symptom checker’s advice (19/34, 56%). Propensity to trust and eHealth literacy were associated with increased subjective trust in the symptom checker (propensity to trust b=0.25; eHealth literacy b=0.2), whereas sociodemographic variables showed no such link with either subjective or behavioral trust. Conclusions Contrary to our expectation, neither the anthropomorphic framing nor the emphasis on AI increased trust in symptom checker advice compared with that of a neutral control condition. However, independent of the interface, most participants trusted the mock app’s advice, even when they were very certain of their own assessment. Thus, the question arises as to whether laypersons use such symptom checkers as substitutes rather than as aids in their own decision-making. With trust in symptom checkers already high at baseline, the benefit of symptom checkers depends on interface designs that enable users to adequately calibrate their trust levels during usage. Trial Registration Deutsches Register Klinischer Studien DRKS00028561; https://tinyurl.com/rv4utcfb (retrospectively registered).

Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review

“…The ability of OSCs to change or direct the behavior of users through the provision of triage recommendations remains unclear; this has significant implications for their likely impact on health service use and health outcomes. Current evidence on user compliance remains scarce and inconsistent: studies with relatively positive results tend to be limited to users’ intention to follow the recommendation [ 26 , 33 ] or to experiments with vignettes instead of users’ own symptoms [ 34 ]. Meanwhile, 2 other studies reported less encouraging results, with few users following the ED visit advice [ 8 ] or a majority of patients in a primary care waiting room not changing their decision to see the GP despite the OSC’s alternative advice to wait and self-care [ 35 ].…”

Only the anxious ones? Identifying characteristics of symptom checker app users: a cross-sectional survey

“…Such difficulties may initiate unnecessary healthcare-seeking behavior, although the evidence remains inconclusive [ 13 ]. Additionally, increased eHealth literacy may lead to greater subjective trust in SCAs and the ability to critically evaluate their recommendations, but not necessarily to a change in actual trust-based behavior [ 14 ]. Lastly, user attitudes toward technology play a significant role, with “tech seekers” being more likely to use SCAs in the future compared to “tech rejectors” and “unsure acceptors” [ 15 ].…”

Assessing the accuracy of a digital symptom checker tool for suggestion of reproductive health conditions: a clinical vignettes study

“…A recent review of online symptom checkers found diagnostic accuracy of the primary diagnosis varied from 19-38% and triage accuracy ranged from 49-90% 52 . Even though information on their development and validation is limited and its reliability in question 45,51 , trust in symptom checker apps is high among laypersons 53 .…”