Using Artificial Intelligence (Watson for Oncology) for Treatment Recommendations Amongst Chinese Patients with Lung Cancer: Feasibility Study

Abstract

BackgroundArtificial intelligence (AI) is developing quickly in the medical field and can benefit both medical staff and patients. The clinical decision support system Watson for Oncology (WFO) is an outstanding representative AI in the medical field, and it can provide to cancer patients prompt treatment recommendations comparable with ones made by expert oncologists. WFO is increasingly being used in China, but limited reports on whether WFO is suitable for Chinese patients, especially patients with lung cancer, exist. Here, we report a retrospective study based on the consistency between the lung cancer treatment recommendations made for the same patient by WFO and by the multidisciplinary team at our center.ObjectiveThe aim of this study was to explore the feasibility of using WFO for lung cancer cases in China and to ascertain ways to make WFO more suitable for Chinese patients with lung cancer.MethodsWe selected all lung cancer patients who were hospitalized and received antitumor treatment for the first time at the Second Xiangya Hospital Cancer Center from September to December 2017 (N=182). WFO made treatment recommendations for all supported cases (n=149). If the actual therapeutic regimen (administered by our multidisciplinary team) was recommended or for consideration according to WFO, we defined the recommendations as consistent; if the actual therapeutic regimen was not recommended by WFO or if WFO did not provide the same treatment option, we defined the recommendations as inconsistent. Blinded second round reviews were performed by our multidisciplinary team to reassess the incongruent cases.ResultsWFO did not support 18.1% (33/182) of recommendations among all cases. Of the 149 supported cases, 65.8% (98/149) received recommendations that were consistent with the recommendations of our team. Logistic regression analysis showed that pathological type and staging had significant effects on consistency (P=.004, odds ratio [OR] 0.09, 95% CI 0.02-0.45 and P<.001, OR 9.5, 95% CI 3.4-26.1, respectively). Age, gender, and presence of epidermal growth factor receptor gene mutations had no effect on consistency. In 82% (42/51) of the inconsistent cases, our team administered two China-specific treatments, which were different from the recommendations made by WFO but led to excellent outcomes.ConclusionsIn China, most of the treatment recommendations of WFO are consistent with the recommendations of the expert group, although a relatively high proportion of cases are still not supported by WFO. Therefore, WFO cannot currently replace oncologists. WFO can improve the efficiency of clinical work by providing assistance to doctors, but it needs to learn the regional characteristics of patients to improve its assistive ability.

Concordance in postsurgical radioactive iodine therapy recommendations between Watson for Oncology and clinical practice in patients with differentiated thyroid carcinoma

“…This suggests that WFO recommendations may be based on the treatment preferences of a small number of physicians who provided the data, and not actual insights gained by analyzing actual patients. Previous studies of this subject demonstrated that the concordance rate varied by countries and cancer types . As mentioned earlier, a retrospective study of patients in India who were diagnosed with breast cancer demonstrated a treatment recommendation concordance rate of 93% between WFO and the multidisciplinary tumor board .…”

“…A double‐blind study from India found that breast cancer treatment recommendations made by both WFO and a multidisciplinary tumor board were highly concordant . In another retrospective study of Chinese patients with lung cancer, WFO did not support approximately 18% of the cases . Among the supported cases, the rate of treatment concordance between WFO and the expert group was 66% .…”

“…Watson for Oncology (WFO) is an artificial intelligence–based clinical decision support system for anticancer therapy selection . It provides prompt evidence‐based and personalized treatment options after processing the patient's data .…”

“…WFO reduces the ambiguity in optimal therapy decision making that can manifest when personal preferences or cognitive biases dominate individual decisions, thus paving the way for standardized treatments . To enable generalized screening, it is critical to achieve concordance in the recommendations rendered by WFO and those offered by physicians to verify the accuracy of the former . A double‐blind study from India found that breast cancer treatment recommendations made by both WFO and a multidisciplinary tumor board were highly concordant .…”

“…In another retrospective study of Chinese patients with lung cancer, WFO did not support approximately 18% of the cases . Among the supported cases, the rate of treatment concordance between WFO and the expert group was 66% . A recent study that encompassed patients from China diagnosed with several types of cancer demonstrated that the concordance between WFO's and physicians' decisions was not as high as previously reported in other countries; moreover, concordance rates varied by cancer type, ranging from 12% to 96% …”