A tool for patient‐specific prediction of delivery discrepancies in machine parameters using trajectory log files

Abstract

Purpose Multileaf collimator (MLC) delivery discrepancy between planned and actual (delivered) positions have detrimental effect on the accuracy of dose distributions for both IMRT and VMAT. In this study, we evaluated the consistency of MLC delivery discrepancies over the course of treatment and over time to verify that a predictive machine learning model would be applicable throughout the course of treatment. Next, the MLC and gantry positions recorded in prior trajectory log files were analyzed to build a machine learning algorithm to predict MLC positional discrepancies during delivery for a new treatment plan. An open source tool was developed and released to predict the MLC positional discrepancies at treatment delivery for any given plan. Methods Trajectory log files of 142 IMRT plans and 125 VMAT plans from 9 Varian TrueBeam linear accelerators were collected and analyzed. The consistency of delivery discrepancy over patient‐specific quality assurance (QA) and patient treatment deliveries was evaluated. Data were binned by treatment site and machine type to determine their relationship with MLC and gantry angle discrepancies. Motion‐related parameters including MLC velocity, MLC acceleration, control point, dose rate, and gravity vector, gantry velocity and gantry acceleration, where applicable, were analyzed to evaluate correlations with MLC and gantry discrepancies. Several regression models, such as simple/multiple linear regression, decision tree, and ensemble method (boosted tree and bagged tree model) were used to develop a machine learning algorithm to predict MLC discrepancy based on MLC motion parameters. Results MLC discrepancies at patient‐specific QA differed from those at patient treatment deliveries by a small (mean = 0.0021 ± 0.0036 mm, P = 0.0089 for IMRT; mean = 0.0010 ± 0.0016 mm, P = 0.0003 for VMAT) but statistically significant amount, likely due to setting the gantry angle to zero for QA in IMRT. MLC motion parameters, MLC velocity and gravity vector, showed significant correlation (P < 0.001) with MLC discrepancy, especially MLC velocity, which had an approximately linear relationship (slope = −0.0027, P < 0.001, R2 = 0.79). Incorporating MLC motion parameters, the final generalized model trained by data from all linear accelerators can predict MLC discrepancy to a high degree of accuracy with high correlation (R2 = 0.86) between predicted and actual MLC discrepancies. The same prediction results were found across different treatment sites and linear accelerators. Conclusion We have developed a machine learning model using trajectory log files to predict the MLC discrepancies during delivery. This model has been a released as a research tool in which a DICOM‐RT with predicted MLC positions can be generated using the original DICOM‐RT file as input. This tool can be used to simulate radiotherapy treatment delivery and may be useful for studies evaluating plan robustness and dosimetric uncertainties from treatment delivery.

Applications of machine and deep learning to patient‐specific IMRT/VMAT quality assurance

“…treatment site) can be used as model input features. Various ML and DL models were trained to map the plan complexity features alone or with other dosimetric features 33–35,38,41,43–48,51 . Complexity metrics can be related to the QA outcome results, offering a troubleshooting method in case a plan fails the QA.…”

“…Their model revealed high prediction accuracy for all individual MLCs with a maximum root MSE (RMSE) of 0.0097 mm between the predicted and the actual leaf positions. Chuang et al 43 studied various regression ML models including simple and multivariate linear regressions, decision tree, and ensemble method (boosted tree and bagged tree model) using trajectory log files data to predict thee MLC positional errors for IMRT and VMAT treatment. The results showed that the boosted tree model had the best performance with RMSE of 0.0085 mm.…”

“…Regardless of the prediction performance, ML and DL models that are trained on large datasets 38 , 40 , 47 , 49 provide more reliable results. The results reported by Carlson et al, 44 Osman et al, 41 and Chuang et al 43 used large data sets, ranging from hundred‐thousand to millions data samples, for training various ML models. Compared to ML, a DL model needs larger datasets to properly train and to optimize its hyper‐parameters.…”

“…Various ML and DL models were trained to map the plan complexity features alone or with other dosimetric features. 33 , 34 , 35 , 38 , 41 , 43 , 44 , 45 , 46 , 47 , 48 , 51 Complexity metrics can be related to the QA outcome results, offering a troubleshooting method in case a plan fails the QA. Complexity metrics identified as important features include MU value, 34 , 35 , 38 , 45 , 47 , 48 beam irregularity factor, 34 , 38 , 45 , 48 aperture size/area, 34 , 38 , 45 , 46 , 48 , 49 and others (Table 3 ).…”

Artificial intelligence for prediction of measurement‐based patient‐specific quality assurance is ready for prime time

“…For instance, uncertainties in dose calculation can now be quantified directly via Monte Carlo (MC)‐based independent calculation 13 . Uncertainties in machine parameters at delivery are quantifiable via trajectory files, and, if desired, performance for individual plans can be accurately predicted using AI 14,15 . This approach has major advantages: (a) it solves the clinical relevance problem since results can be used to directly and unambiguously calculate effect on patient dose‐volume histogram (DVH), and (b) it decouples clinically irrelevant uncertainties related to the PSQA process.…”