Innovative integration of augmented reality and optical surface imaging: A coarse‐to‐precise system for radiotherapy positioning

Abstract

Background Traditional methods of radiotherapy positioning have shortcomings such as fragile skin‐markers, additional doses, and lack of information integration. Emerging technologies may provide alternatives for the relevant clinical practice. Purpose To propose a noninvasive radiotherapy positioning system integrating augmented reality (AR) and optical surface, and to evaluate its feasibility in clinical workflow. Methods AR and structured light‐based surface were integrated to implement the coarse‐to‐precise positioning through two coherent steps, the AR‐based coarse guidance and the optical surface‐based precise verification. To implement quality assurance, recognition of face and pattern was used for patient authentication, case association, and accessory validation in AR scenes. The holographic images reconstructed from simulation computed tomography (CT) images, guided the initial posture correction by virtual‐real alignment. The point clouds of body surface were fused, with the calibration and pose estimation of structured light cameras, and segmented according to the preset regions of interest (ROIs). The global‐to‐local registration for cross‐source point clouds was achieved to calculate couch shifts in six degrees‐of‐freedom (DoF), which were ultimately transmitted to AR scenes. The evaluation based on phantom and human‐body (4 volunteers) included, (i) quality assurance workflow, (ii) errors of both steps and correlation analysis, (iii) receiver operating characteristic (ROC), (iv) distance characteristics of accuracy, and (v) clinical positioning efficiency. Results The maximum errors in phantom evaluation were 3.4 ± 2.5 mm in Vrt and 1.4 ± 1.0° in Pitch for the coarse guidance step, while 1.6 ± 0.9 mm in Vrt and 0.6 ± 0.4° in Pitch for the precise verification step. The Pearson correlation coefficients between precise verification and cone beam CT (CBCT) results were distributed in the interval [0.81, 0.85]. In ROC analysis, the areas under the curve (AUC) were 0.87 and 0.89 for translation and rotation, respectively. In human body‐based evaluation, the errors of thorax and abdomen (T&A) were significantly greater than those of head and neck (H&N) in Vrt (2.6 ± 1.1 vs. 1.7 ± 0.8, p < 0.01), Lng (2.3 ± 1.1 vs. 1.4 ± 0.9, p < 0.01), and Rtn (0.8 ± 0.4 vs. 0.6 ± 0.3, p = 0.01) while relatively similar in Lat (1.8 ± 0.9 vs. 1.7 ± 0.8, p = 0.07). The translation displacement range, after coarse guidance step, required for high accuracy of the optical surface component of the integrated system was 0–42 mm, and the average positioning duration of the integrated system was significantly less than that of conventional workflow (355.7 ± 21.7 vs. 387.7 ± 26.6 s, p < 0.01). Conclusions The combination of AR and optical surface has utility and feasibility for patient positioning, in terms of both safety and accuracy.

Feasibility evaluation of radiotherapy positioning system guided by augmented reality and point cloud registration

“…Nevertheless, the difference lies in our further exploration of the application of ICP, including investigating the sampling method, discrete point elimination, ICP type, etc., to enhance the accuracy and stability of registration in medical phantoms. In similar studies using Point-to-Plane ICP for radiotherapy positioning, 21 the optimal positioning error for human thoracic and abdominal phantom was still above 1 mm, while that in our phantom study was in sub-millimeter, significantly improving the accuracy, which further confirmed the necessity of thoroughly exploring the steps of PCR. Furthermore, the optimal steps of PCR may be different under different clinical situations, which requires us to explore the balance between accuracy and speed by combining different clinical situations.…”

“… 6 Therefore, to meet the accuracy and stability requirements of radiotherapy positioning, we used an HD camera instead of an HMD, which can provide more stable recognition results and also enables AR surveillance. Compared with other studies using AR, 6 , 21 , 25 the error of the proposed system is significantly lower, especially the displacement deviations in sub‐millimeter, which is comparable to the advanced SGRT. 26 …”

“…26 Although the application of PCR has become increasingly extensive in the medical field, [27][28][29] few studies focus on its optimal utilization in radiotherapy. Some studies 21,25 have employed ICP registration for radiotherapy positioning, similar to our approach. Nevertheless, the difference lies in our further exploration of the application of ICP, including investigating the sampling method, discrete point elimination, ICP type, etc., to enhance the accuracy and stability of registration in medical phantoms.…”

“…However, despite the decreased accuracy observed in free-breathing positioning, the results still fall within the clinically acceptable error range and are higher than those reported in similar studies. 6,21,25 Nevertheless, the decrease in the accuracy highlights the need to minimize the effects of breathing-induced deformation during positioning.To address this,it is necessary to optimize the registration algorithm, such as by incorporating a deformation registration step to correct for breathing errors. In addition to considering the impact of breathing.…”

Augmented reality in brachytherapy: A narrative review

“…The study findings demonstrated promising results, with maximum errors of 3.4 mm in coarse guidance and 1.6 mm in precise verification. Pearson’s correlation coefficients between the precise verification and cone-beam CT (CBCT) results were high, indicating the potential of AR in improving patient positioning accuracy [ 23 ].…”