Preferences for Artificial Intelligence Clinicians Before and During the COVID-19 Pandemic: Discrete Choice Experiment and Propensity Score Matching Study

Abstract

Background Artificial intelligence (AI) methods can potentially be used to relieve the pressure that the COVID-19 pandemic has exerted on public health. In cases of medical resource shortages caused by the pandemic, changes in people’s preferences for AI clinicians and traditional clinicians are worth exploring. Objective We aimed to quantify and compare people’s preferences for AI clinicians and traditional clinicians before and during the COVID-19 pandemic, and to assess whether people’s preferences were affected by the pressure of pandemic. Methods We used the propensity score matching method to match two different groups of respondents with similar demographic characteristics. Respondents were recruited in 2017 and 2020. A total of 2048 respondents (2017: n=1520; 2020: n=528) completed the questionnaire and were included in the analysis. Multinomial logit models and latent class models were used to assess people’s preferences for different diagnosis methods. Results In total, 84.7% (1115/1317) of respondents in the 2017 group and 91.3% (482/528) of respondents in the 2020 group were confident that AI diagnosis methods would outperform human clinician diagnosis methods in the future. Both groups of matched respondents believed that the most important attribute of diagnosis was accuracy, and they preferred to receive combined diagnoses from both AI and human clinicians (2017: odds ratio [OR] 1.645, 95% CI 1.535-1.763; P<.001; 2020: OR 1.513, 95% CI 1.413-1.621; P<.001; reference: clinician diagnoses). The latent class model identified three classes with different attribute priorities. In class 1, preferences for combined diagnoses and accuracy remained constant in 2017 and 2020, and high accuracy (eg, 100% accuracy in 2017: OR 1.357, 95% CI 1.164-1.581) was preferred. In class 2, the matched data from 2017 were similar to those from 2020; combined diagnoses from both AI and human clinicians (2017: OR 1.204, 95% CI 1.039-1.394; P=.011; 2020: OR 2.009, 95% CI 1.826-2.211; P<.001; reference: clinician diagnoses) and an outpatient waiting time of 20 minutes (2017: OR 1.349, 95% CI 1.065-1.708; P<.001; 2020: OR 1.488, 95% CI 1.287-1.721; P<.001; reference: 0 minutes) were consistently preferred. In class 3, the respondents in the 2017 and 2020 groups preferred different diagnosis methods; respondents in the 2017 group preferred clinician diagnoses, whereas respondents in the 2020 group preferred AI diagnoses. In the latent class, which was stratified according to sex, all male and female respondents in the 2017 and 2020 groups believed that accuracy was the most important attribute of diagnosis. Conclusions Individuals’ preferences for receiving clinical diagnoses from AI and human clinicians were generally unaffected by the pandemic. Respondents believed that accuracy and expense were the most important attributes of diagnosis. These findings can be used to guide policies that are relevant to the development of AI-based health care.

Perceptions and Needs of Artificial Intelligence in Health Care to Increase Adoption: Scoping Review

“…Sample sizes for studies with human subject responses ranged from 11 to 2780, and secondary data (ie, journal articles and app reviews) ranged from 31 to 1826 [22][23][24]. Interestingly, 19% (5/26) of studies focused on the use of chatbots in health care [23][24][25][26][27] and 31% (8/26) of studies measured acceptability using questionnaires, surveys, interviews [25,26,[28][29][30][31][32][33], and the discrete choice experiment (Multimedia Appendix 4 [22][23][24][25][26][27][28][29][30][31][32]34,36,37,39,[41][42][43][44]47]) [34]. All the studies showed at least moderate acceptability, or >50% of the participants showed acceptance toward the use of AI in health care, albeit only for minor conditions [26].…”

“…Age, IT skills, preference for talking to computers, perceived utility, positive attitude, and perceived trustworthiness were found to be associated with AI acceptability [25,26]. Australia and New Zealand [35] 4 (15) Canada [27,[36][37][38] 6 (23) China [22,32,33,39,40] 1 (4) France [41] 2 (8) India [24,42] 1 (4) Korea [48] 1 (4) Saudi Arabia [29] 1 (4) Switzerland [30] 5 (19) United Kingdom [23,26,31,43,44] 1 (4) United Kingdom, Cyprus, Australia, the Netherlands, Sweden, Spain, United States, and Canada [28] 3 (12) United States [25,45,46] Type of publication 24 (92) Journal papers [22][23][24][25][26][27][28][29][31][32][33][34][35][36][37]…”

“…Some respondents also appreciated the use of AI to provide a second opinion to physicians' diagnoses or evaluations [42,46]. Overall, 12% (3/26) of studies [24,34,45] discussed the potential cost-saving capacity of AI that influences AI acceptability, whereas 4% (1/26) mentioned that the provision of an AI service using IBM Watson caused patients to incur higher treatment costs that did not translate to profits for the hospital after factoring onboarding of the technology [40]. There was a good proportion of representation from the health care and IT staff (53.3%) [27][28][29]36,37,40,42,44] and those from the public, including patients (Multimedia Appendix 3).…”

“…Of the 26 studies, 21 (81%) discussed the respondents' lack of trust in an AI to ensure patient safety while performing its tasks, especially regarding providing accurate information on rare conditions or unexpected situations [22][23][24][25][26][27][29][30][31][32][33][34][35][36][37][38][39][40][41][42]44]. Other concerns were regarding the credibility of AI-based recommendations (eg, whether it was validated by medical professionals) [30,32], maturity in the technology to provide safe and realistic recommendations [22,25], medical liability from the risk of medical errors and bias [26,35,36,44], secondary effects of AI-based diagnoses such as insurance claims [44], and miscommunications [26].…”

“…Most respondents accepted AI in providing general health advice to minor ailments. Most of the responses represented the voices of the public, including patients (66.6%) [22][23][24][25][26][30][31][32]34,35,[37][38][39][40] (Multimedia Appendix 3).…”