A Decision Support Software for AI-Assisted Decision Making in Response-Adaptive Radiotherapy — An Evaluation Study

Niraula, Dipesh; Jin, J.; Dinov, I.D.; Cuneo, K.C.; Jamaluddin, Jehana Bte; Matuszak, M.M.; Haken, R.K. Ten; Naqa, I. El

doi:10.1016/j.ijrobp.2022.07.895

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the response evaluation phase, patients’ treatment response is assessed by comparing pre and during-treatment multi-omics information and in the adaptation phase, a treatment plan is adapted (dose escalation/deescalation). In this study, the clinicians collaborated with ARCliDS 22,25,26 —a software for dynamic decision-making developed using a model-based deep reinforcement learning algorithm. For application in KBR-ART, ARCliDS uses a graphical neural network-based model of radiotherapy environment which defines a patient’s state via a graph of multi-omics features and is capable of assessing treatment response and predicting treatment outcomes.…”

Section: Introductionmentioning

confidence: 99%

Intricacies of Human-AI Interaction in Dynamic Decision-Making for Precision Oncology: A Case Study in Response-Adaptive Radiotherapy

Niraula,

Cuneo,

Dinov

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: Adaptive treatment strategies that can dynamically react to individual cancer progression can provide effective personalized care. Longitudinal multi-omics information, paired with an artificially intelligent clinical decision support system (AI-CDSS) can assist clinicians in determining optimal therapeutic options and treatment adaptations. However, AI-CDSS is not perfectly accurate, as such, clinicians' over/under reliance on AI may lead to unintended consequences, ultimately failing to develop optimal strategies. To investigate such collaborative decision-making process, we conducted a Human-AI interaction case study on response-adaptive radiotherapy (RT). Methods: We designed and conducted a two-phase study for two disease sites and two treatment modalities—adaptive RT for non-small cell lung cancer (NSCLC) and adaptive stereotactic body RT for hepatocellular carcinoma (HCC)—in which clinicians were asked to consider mid-treatment modification of the dose per fraction for a number of retrospective cancer patients without AI-support (Unassisted Phase) and with AI-assistance (AI-assisted Phase). The AI-CDSS graphically presented trade-offs in tumor control and the likelihood of toxicity to organs at risk, provided an optimal recommendation, and associated model uncertainties. In addition, we asked for clinicians' decision confidence level and trust level in individual AI recommendations and encouraged them to provide written remarks. We enrolled 13 evaluators (radiation oncology physicians and residents) from two medical institutions located in two different states, out of which, 4 evaluators volunteered in both NSCLC and HCC studies, resulting in a total of 17 completed evaluations (9 NSCLC, and 8 HCC). To limit the evaluation time to under an hour, we selected 8 treated patients for NSCLC and 9 for HCC, resulting in a total of 144 sets of evaluations (72 from NSCLC and 72 from HCC). Evaluation for each patient consisted of 8 required inputs and 2 optional remarks, resulting in up to a total of 1440 data points. Results: AI-assistance did not homogeneously influence all experts and clinical decisions. From NSCLC cohort, 41 (57%) decisions and from HCC cohort, 34 (47%) decisions were adjusted after AI assistance. Two evaluations (12%) from the NSCLC cohort had zero decision adjustments, while the remaining 15 (88%) evaluations resulted in at least two decision adjustments. Decision adjustment level positively correlated with dissimilarity in decision-making with AI [NSCLC: ρ = 0.53 (p < 0.001); HCC: ρ = 0.60 (p < 0.001)] indicating that evaluators adjusted their decision closer towards AI recommendation. Agreement with AI-recommendation positively correlated with AI Trust Level [NSCLC: ρ = 0.59 (p < 0.001); HCC: ρ = 0.7 (p < 0.001)] indicating that evaluators followed AI's recommendation if they agreed with that recommendation. The correlation between decision confidence changes and decision adjustment level showed an opposite trend [NSCLC: ρ = -0.24 (p = 0.045), HCC: ρ = 0.28 (p = 0.017)] reflecting the difference in behavior due to underlying differences in disease type and treatment modality. Decision confidence positively correlated with the closeness of decisions to the standard of care (NSCLC: 2 Gy/fx; HCC: 10 Gy/fx) indicating that evaluators were generally more confident in prescribing dose fractionations more similar to those used in standard clinical practice. Inter-evaluator agreement increased with AI-assistance indicating that AI-assistance can decrease inter-physician variability. The majority of decisions were adjusted to achieve higher tumor control in NSCLC and lower normal tissue complications in HCC. Analysis of evaluators' remarks indicated concerns for organs at risk and RT outcome estimates as important decision-making factors. Conclusions: Human-AI interaction depends on the complex interrelationship between expert's prior knowledge and preferences, patient's state, disease site, treatment modality, model transparency, and AI's learned behavior and biases. The collaborative decision-making process can be summarized as follows: (i) some clinicians may not believe in an AI system, completely disregarding its recommendation, (ii) some clinicians may believe in the AI system but will critically analyze its recommendations on a case-by-case basis; (iii) when a clinician finds that the AI recommendation indicates the possibility for better outcomes they will adjust their decisions accordingly; and (iv) When a clinician finds that the AI recommendation indicate a worse possible outcome they will disregard it and seek their own alternative approach.

show abstract

Section: Introductionmentioning

confidence: 99%

Intricacies of Human-AI Interaction in Dynamic Decision-Making for Precision Oncology: A Case Study in Response-Adaptive Radiotherapy

Niraula,

Cuneo,

Dinov

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration

Wei,

Niraula,

Gates

et al. 2023

The British Journal of Radiology

View full text Add to dashboard Cite

Multiomics data including imaging radiomics and various types of molecular biomarkers have been increasingly investigated for better diagnosis and therapy in the era of precision oncology. Artificial intelligence (AI) including machine learning (ML) and deep learning (DL) techniques combined with the exponential growth of multiomics data may have great potential to revolutionize cancer subtyping, risk stratification, prognostication, prediction and clinical decision-making. In this article, we first present different categories of multiomics data and their roles in diagnosis and therapy. Second, AI-based data fusion methods and modeling methods as well as different validation schemes are illustrated. Third, the applications and examples of multiomics research in oncology are demonstrated. Finally, the challenges regarding the heterogeneity data set, availability of omics data, and validation of the research are discussed. The transition of multiomics research to real clinics still requires consistent efforts in standardizing omics data collection and analysis, building computational infrastructure for data sharing and storing, developing advanced methods to improve data fusion and interpretability, and ultimately, conducting large-scale prospective clinical trials to fill the gap between study findings and clinical benefits.

show abstract

A Decision Support Software for AI-Assisted Decision Making in Response-Adaptive Radiotherapy — An Evaluation Study

Cited by 2 publications

References 0 publications

Intricacies of Human-AI Interaction in Dynamic Decision-Making for Precision Oncology: A Case Study in Response-Adaptive Radiotherapy

Intricacies of Human-AI Interaction in Dynamic Decision-Making for Precision Oncology: A Case Study in Response-Adaptive Radiotherapy

Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration

Contact Info

Product

Resources

About