Concordance assessment of a clinical decision support software in patients with solid tumors.

Suwanrusme, Harit; Issarachai, Surasit; Umsawasdi, Theera; Suwanvecho, Suthida; Decha, Wimolrat; Dankwa‐Mullan, Irene; Wang, Ching-Kun; Urman, Alexandra; Kiatikajornthada, Narongsak

doi:10.1200/jco.2018.36.15_suppl.e18584

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study, similar to those conducted in Asia, demonstrates that the cognitive computer platform IBM WfO is able to render cancer treatment options in concordance with expert opinions. [12][13][14][15][16][17][18][19][20][21] Specifically, in this USA based study 3 blinded oncologists who specialize in the care of breast cancer patients chose treatment strategies in agreement with WfO acceptable options 87.9% of the time, and chose a preferred option in 78.5% of cases. The oncologist who had previously worked at MSKCC more often agreed with the MSKCC trained WfO system suggesting a possible center effect.…”

Section: Discussionmentioning

confidence: 99%

“…An expansion of the 2016 Thailand study conducted in 2017 for a total of 590 cases found the overall concordance rate for the retrospective cases was 81%; 78% for bladder, 70% for breast, 56% for cervical, 90% for colon, 82% for gastric, 93% for lung, 79% for ovarian, 82% for prostate and 94% for rectal. Similarly, the overall concordance rate for the prospective cases was 80%; 75% for bladder, 72% for breast, 78% for cervical, 95% for colon, 71% for gastric, 87% for lung, 63% for ovarian, 86% for prostate and 76% for rectal cancers . A review in China noted recommendations were concordant in 79.0% of breast cancers and 96.9% in lung cancers, with the main reason for disagreement being economic constraints .…”

Section: Introductionmentioning

confidence: 93%

“…Similarly, the overall concordance rate for the prospective cases was 80%; 75% for bladder, 72% for breast, 78% for cervical, 95% for colon, 71% for gastric, 87% for lung, 63% for ovarian, 86% for prostate and 76% for rectal cancers. 15 A review in China noted recommendations were concordant in 79.0% of breast cancers and 96.9% in lung cancers, with the main reason for disagreement being economic constraints. 16,17 A recent review in China for Stage II colon cancer patients found overall concordance was 89.1% with high-risk subgroup results ranging from 87.5% (P = .68) in T4 primary to 92.7% (P = .02) in poorly differentiated histology.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Differential impact of cognitive computing augmented by real world evidence on novice and expert oncologists

McNamara

Goldberg²,

Latts

et al. 2019

Cancer Medicine

Self Cite

View full text Add to dashboard Cite

Introduction Cognitive computing point‐of‐care decision support tools which ingest patient attributes from electronic health records and display treatment options based on expert training and medical literature, supplemented by real world evidence (RWE), might prove useful to expert and novice oncologists. The concordance of augmented intelligence systems with best medical practices and potential influences on physician behavior remain unknown. Methods Electronic health records from 88 breast cancer patients evaluated at a USA tertiary care center were presented to subspecialist experts and oncologists focusing on other disease states with and without reviewing the IBM Watson for Oncology with Cota RWE platform. Results The cognitive computing “recommended” option was concordant with selection by breast cancer experts in 78.5% and “for consideration” option was selected in 9.4%, yielding agreements in 87.9%. Fifty‐nine percent of non‐concordant responses were generated from 8% of cases. In the Cota observational database 69.3% of matched controls were treated with “recommended,” 11.4% “for consideration”, and 19.3% “not recommended.” Without guidance from Watson for Oncology (WfO)/Cota RWE, novice oncologists chose 75.5% recommended/for consideration treatments which improved to 95.3% with WfO/Cota RWE. The novices were more likely than experts to choose a non‐recommended option (P < .01) without WfO/Cota RWE and changed decisions in 39% cases. Conclusions Watson for Oncology with Cota RWE options were largely concordant with disease expert judged best oncology practices, and was able to improve treatment decisions among breast cancer novices. The observation that nearly a fifth of patients with similar disease characteristics received non‐recommended options in a real world database highlights a need for decision support.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 93%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Differential impact of cognitive computing augmented by real world evidence on novice and expert oncologists

McNamara

Goldberg²,

Latts

et al. 2019

Cancer Medicine

Self Cite

View full text Add to dashboard Cite

show abstract

“…According to system use case and country, we identi ed a study conducted at a tertiary cancer center in India with a higher breast cancer treatment decision concordance between the CDSS and MTB, 25 as compared to similar use case studies in China. 23,24,26,27 Likewise, individual clinicians in Thailand had higher concordance with the CDSS in 3 studies 28,30,32 as compared to 2 large individual clinicians studies from China. 29,31 Differences in breast cancer treatment decision concordance between the system and individual clinicians or MTBs in different countries are multifactorial and likely explained by differences in oncology practice patterns at the institutional and national levels.…”

Section: Concordance In Different Countries and Breast Cancer Subgroupsmentioning

confidence: 96%

A Targeted Review of Breast Cancer Studies of Concordance for an Internationally-Implemented Artificially Intelligent Clinical Decision-Support System

Arriaga

Hekmat

Draulis

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Breast cancer has the highest incidence and is the leading cause of cancer-related mortality among women worldwide. IBM Watson® for Oncology (WfO), an artificial intelligence-based clinical decision-support system, provides therapeutic options for consideration to cancer-treating physicians. We conducted a targeted review of studies evaluating concordance of therapeutic options offered by the system with treatment decisions by practicing clinicians in breast cancer. Methods: PubMed, EMBASE, Cochrane, trial registers, conference abstracts, and an internal publication database were searched to identify studies evaluating the concordance of system-generated therapeutic options with treatment decisions by individual clinicians and multidisciplinary tumor boards for breast cancer patients reported in peer-reviewed abstracts or papers published in English between 01/01/2015 and 11/15/2019.Results: Ten breast cancer concordance studies (4703 patients) that met the inclusion criteria were identified and analyzed; the identified studies were from China, India, and Thailand. The weighted mean concordance for all studies was 67.4% (SD 16.0%, range 55.0% - 98.0%). The weighted mean concordance of the system with multidisciplinary tumor boards was 88.2%, (SD 9.7%, range 76.5% - 98.0%), which was substantially higher than concordance between the system and individual clinicians (61.5% , SD 10.1%, range 55.0% -76.0%).Conclusion: Concordance between system-generated therapeutic options and treatment decisions of multidisciplinary tumor boards or individual clinicians for breast cancer demonstrated overall agreement between the system and decisions of practicing cancer-treating physicians in China, India and Thailand. As multidisciplinary tumor boards may lead to higher quality clinical decision-making compared to those of individual clinicians in practice, the relatively higher concordance of the system with multidisciplinary tumor boards suggests a role for clinical decision support to inform clinicians of evidence-informed treatment options.

show abstract

Artificial Intelligence Treatment Decision Support For Complex Breast Cancer Among Oncologists With Varying Expertise

Sepúlveda

Jiang

et al. 2019

JCO Clinical Cancer Informatics

View full text Add to dashboard Cite

PURPOSE The aim of the current study was to assess treatment concordance and adherence to National Comprehensive Cancer Network breast cancer treatment guidelines between oncologists and an artificial intelligence advisory tool. PATIENTS AND METHODS Study cases of patients (N = 1,977) who were at high risk for recurrence or who had metastatic disease and cell types for which the advisory tool was trained were obtained from the Chinese Society for Clinical Oncology cancer database (2012 to 2017). A cross-sectional observational study was performed to examine treatment concordance and guideline adherence among an artificial intelligence advisory tool and 10 oncologists with varying expertise—three fellows, four attending physicians, and three chief physicians. In a blinded fashion, each oncologist provided treatment advice on an average of 198 cases and the advisory tool on all cases (N = 1,977). Results are reported as rates and logistic regression odds ratios. RESULTS Concordance for the recommended treatment was 0.56 for all physicians and higher for fellows compared with chief and attending physicians (0.68 v 0.54; 0.49; P = .001). Concordance differed by hormone receptor subtype–TNM stage, with the lowest for hormone receptor–positive human epidermal growth factor receptor 2/neu-positive cancers (0.48) and highest for triple-negative breast cancers (0.71) across most TNM stages. Adherence to National Comprehensive Cancer Network guidelines was higher for oncologists compared with the advisory tool (0.96 v 0.82; P < .003) and lower for fellows compared with attending physicians (0.93 v 0.98; 0.96; P = .04). CONCLUSION Study findings reflect a complex breast cancer case mix, the limits of medical knowledge regarding optimum treatment, clinician practice patterns, and use of a tool that reflects expertise from one cancer center. Additional research in different practice settings is needed to understand the tool’s scalability and its impact on treatment decisions and clinical and health services outcomes.

show abstract

Concordance assessment of a clinical decision support software in patients with solid tumors.

Cited by 4 publications

References 0 publications

Differential impact of cognitive computing augmented by real world evidence on novice and expert oncologists

Differential impact of cognitive computing augmented by real world evidence on novice and expert oncologists

A Targeted Review of Breast Cancer Studies of Concordance for an Internationally-Implemented Artificially Intelligent Clinical Decision-Support System

Artificial Intelligence Treatment Decision Support For Complex Breast Cancer Among Oncologists With Varying Expertise

Contact Info

Product

Resources

About