Diagnostic Performance by Medical Students Working Individually or in Teams

Hautz, Wolf E.; Kämmer, Juliane E.; Schauber, Stefan K.; Spies, Claudia; Gaissmaier, Wolfgang

doi:10.1001/jama.2014.15770

Cited by 72 publications

(88 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result can be considered in agreement with several studies on different medical issues showing that predictor's confidence correlates very well with the correctness of the prediction ( Detsky et al , 2017; Hautz et al , 2015; Kämmer et al , 2017; Kurvers et al , 2016). Indeed, the concordance of different members of a given group (students or runs of the random forest model) can be taken as indicating that the agent is "sure" of the forecast.…”

Section: Discussionsupporting

confidence: 92%

“…However, when expert people are involved, even small groups can outperform the best among them, at least when a yes/no answer to well-defined diagnostic questions is requested based on radiographic/ histological images, ( Kurvers et al , 2016; Sonabend et al , 2017; Wolf et al , 2015). Studies with medical students show that working in pairs ameliorates diagnostic ability, with further improvements when group size increases ( Hautz et al , 2015; Kämmer et al , 2017), in line with the core idea of Collective intelligence.…”

Section: Introductionmentioning

confidence: 66%

See 1 more Smart Citation

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Tacchella¹,

Romano

Ferraldeschi

et al. 2017

F1000Res

View full text Add to dashboard Cite

Background: Multiple sclerosis has an extremely variable natural course. In most patients, disease starts with a relapsing-remitting (RR) phase, which proceeds to a secondary progressive (SP) form. The duration of the RR phase is hard to predict, and to date predictions on the rate of disease progression remain suboptimal. This limits the opportunity to tailor therapy on an individual patient's prognosis, in spite of the choice of several therapeutic options. Approaches to improve clinical decisions, such as collective intelligence of human groups and machine learning algorithms are widely investigated. Methods: Medical students and a machine learning algorithm predicted the course of disease on the basis of randomly chosen clinical records of patients that attended at the Multiple Sclerosis service of Sant'Andrea hospital in Rome. Results: A significant improvement of predictive ability was obtained when predictions were combined with a weight that depends on the consistence of human (or algorithm) forecasts on a given clinical record. Conclusions: In this work we present proof-of-principle that human-machine hybrid predictions yield better prognoses than machine learning algorithms or groups of humans alone. To strengthen this preliminary result, we propose a crowdsourcing initiative to collect prognoses by physicians on an expanded set of patients.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Introductionmentioning

confidence: 66%

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Tacchella¹,

Romano

Ferraldeschi

et al. 2017

F1000Res

View full text Add to dashboard Cite

show abstract

“…8,9,10,11 To date, the existing evidence on the role of collective intelligence in general clinical diagnosis has been limited to single-center studies of medical students. 11,16 However, in most medical specialties, the diagnostic process requires combining many pieces of information while considering multiple diagnoses. The present study focused on general clinical diagnosis rather than binary decisions in narrow medical domains and, to our knowledge, is the largest study to date of collective intelligence in medicine.…”

Section: Introductionmentioning

confidence: 99%

Comparative Accuracy of Diagnosis by Collective Intelligence of Multiple Physicians vs Individual Physicians

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Reducing the frequency of diagnostic errors is thus a major step toward improving health care (24,25). Previous research on collective intelligence in medical diagnostics has yielded conflicting results: Some studies have found that group decision making boosts diagnostic accuracy (9,12,26,27), whereas others have found null or even detrimental effects (28,29).…”

mentioning

confidence: 99%

Boosting medical diagnostics by pooling independent judgments

Kurvers

Herzog

Hertwig

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

157

140

View full text Add to dashboard Cite

Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decision making in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involving more than 140 doctors making more than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.collective intelligence | groups | medical diagnostics | dermatology | mammography C ollective intelligence, that is, the ability of groups to outperform individual decision makers when solving complex cognitive problems, is a powerful approach for boosting decision accuracy (1-7). However, despite its potential to boost accuracy in a wide range of contexts, including lie detection, political forecasting, investment decisions, and medical decision making (8-14), little is known about the conditions that underlie the emergence of collective intelligence in real-world domains. Which features of decision makers and decision contexts favor the emergence of collective intelligence? Which decision-making rules permit this potential to be harnessed? We here provide answers to these important questions in the domain of medical diagnostics.Our work builds on recent findings on combining decisions, a research paradigm known as "two heads better than one" (15)(16)(17)(18)(19)(20). In their seminal study, Bahrami et al. (15) showed that two individuals permitted to communicate freely while engaging in a visual perception task, achieved better results than the better of the two did alone. Koriat (17) subsequently demonstrated that this collective intelligence effect also emerges in the absence of communication when the "maximum-confidence slating algorithm" (hereafter called confidence rule) is used and th...

show abstract

Diagnostic Performance by Medical Students Working Individually or in Teams

Cited by 72 publications

References 4 publications

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Comparative Accuracy of Diagnosis by Collective Intelligence of Multiple Physicians vs Individual Physicians

Boosting medical diagnostics by pooling independent judgments

Contact Info

Product

Resources

About