Context-aware, reference-free local motion metric for CBCT deformable motion compensation

Huang, Heyuan; Siewerdsen, Jeffrey H.; Zbijewski, Wojciech; Weiss, Clifford R.; Unberath, Mathias; Sisniega, Alejandro

doi:10.1117/12.2646857

Cited by 4 publications

(13 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The deformable autofocus algorithm used in this work, illustrated in Figure 3, was based on previous work on deformable motion estimation with assumptions of local rigidity within small ROIs placed throughout the FOV. 11,47,48 The autofocus motion estimation method acts on a set of N R ROIs of arbitrary size and positioned at arbitrary locations ⃗ r n {n = 1, …, N R } within the CBCT volume. In line with previous approaches, we assume that the motion trajectory within any ROI can be considered rigid and modeled as a time-dependent vector, with three degrees-of -freedom, and with a temporal variation following a cubic b-spline basis with N t temporal knots.…”

Section: Deep Autofocus Motion Compensation With Vif DLmentioning

confidence: 99%

Deformable motion compensation in interventional cone‐beam CT with a context‐aware learned autofocus metric

Huang,

Liu,

Siewerdsen

et al. 2024

Medical Physics

Self Cite

View full text Add to dashboard Cite

PurposeInterventional Cone‐Beam CT (CBCT) offers 3D visualization of soft‐tissue and vascular anatomy, enabling 3D guidance of abdominal interventions. However, its long acquisition time makes CBCT susceptible to patient motion. Image‐based autofocus offers a suitable platform for compensation of deformable motion in CBCT, but it relies on handcrafted motion metrics based on first‐order image properties and that lack awareness of the underlying anatomy. This work proposes a data‐driven approach to motion quantification via a learned, context‐aware, deformable metric, , that quantifies the amount of motion degradation as well as the realism of the structural anatomical content in the image.MethodsThe proposed was modeled as a deep convolutional neural network (CNN) trained to recreate a reference‐based structural similarity metric—visual information fidelity (VIF). The deep CNN acted on motion‐corrupted images, providing an estimation of the spatial VIF map that would be obtained against a motion‐free reference, capturing motion distortion, and anatomic plausibility. The deep CNN featured a multi‐branch architecture with a high‐resolution branch for estimation of voxel‐wise VIF on a small volume of interest. A second contextual, low‐resolution branch provided features associated to anatomical context for disentanglement of motion effects and anatomical appearance. The deep CNN was trained on paired motion‐free and motion‐corrupted data obtained with a high‐fidelity forward projection model for a protocol involving 120 kV and 9.90 mGy. The performance of was evaluated via metrics of correlation with ground truth and with the underlying deformable motion field in simulated data with deformable motion fields with amplitude ranging from 5 to 20 mm and frequency from 2.4 up to 4 cycles/scan. Robustness to variation in tissue contrast and noise levels was assessed in simulation studies with varying beam energy (90–120 kV) and dose (1.19–39.59 mGy). Further validation was obtained on experimental studies with a deformable phantom. Final validation was obtained via integration of on an autofocus compensation framework, applied to motion compensation on experimental datasets and evaluated via metric of spatial resolution on soft‐tissue boundaries and sharpness of contrast‐enhanced vascularity.ResultsThe magnitude and spatial map of showed consistent and high correlation levels with the ground truth in both simulation and real data, yielding average normalized cross correlation (NCC) values of 0.95 and 0.88, respectively. Similarly, achieved good correlation values with the underlying motion field, with average NCC of 0.90. In experimental phantom studies, properly reflects the change in motion amplitudes and frequencies: voxel‐wise averaging of the local across the full reconstructed volume yielded an average value of 0.69 for the case with mild motion (2 mm, 12 cycles/scan) and 0.29 for the case with severe motion (12 mm, 6 cycles/scan). Autofocus motion compensation using resulted in noticeable mitigation of motion artifacts and improved spatial resolution of soft tissue and high‐contrast structures, resulting in reduction of edge spread function width of 8.78% and 9.20%, respectively. Motion compensation also increased the conspicuity of contrast‐enhanced vascularity, reflected in an increase of 9.64% in vessel sharpness.ConclusionThe proposed , featuring a novel context‐aware architecture, demonstrated its capacity as a reference‐free surrogate of structural similarity to quantify motion‐induced degradation of image quality and anatomical plausibility of image content. The validation studies showed robust performance across motion patterns, x‐ray techniques, and anatomical instances. The proposed anatomy‐ and context‐aware metric poses a powerful alternative to conventional motion estimation metrics, and a step forward for application of deep autofocus motion compensation for guidance in clinical interventional procedures.

show abstract

Section: Deep Autofocus Motion Compensation With Vif DLmentioning

confidence: 99%

Deformable motion compensation in interventional cone‐beam CT with a context‐aware learned autofocus metric

Huang,

Liu,

Siewerdsen

et al. 2024

Medical Physics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The performance on the 3D masking task was assessed with 3D DICE scores against the ground truth 3D positions of the target structures. To obtain synthetic motion corrupted images, we introduced to each volume of the test set a smooth, deformable, timevarying MVF with [4,6,8] mm maximum motion amplitude, positioned randomly in the upper abdomen (resembling clinically observed motion 1 ). The temporal trajectory was set as a phase-shifted sinusoid with frequency ranging from 1 to 1.5 cycles/scan.…”

Section: Training and Experimental Validationmentioning

confidence: 99%

“…Prior work towards deformable motion compensation includes image-based autofocus methods that optimize handcrafted sharpness metrics that are agnostic to the underlying anatomy 2 . Recent research on autofocus metrics yielded deep autofocus methods, based on learned-based image appearance models that combine measures of image sharpness and realism of anatomical image content 3,4 . While deep autofocus integrates anatomical knowledge via data-driven metrics, those methods aim at achieving uniform compensation across tissues, disregarding the imaging task (i.e., visualization of contrast-enhanced vascular anatomy in this work).…”

Section: Introductionmentioning

confidence: 99%

Deformable motion compensation for intraprocedural vascular cone-beam CT with sequential projection domain targeting and vessel-enhancing autofocus

Huang

et al. 2023

Medical Imaging 2023: Image-Guided Procedures, Robotic Interventions, and Modeling

View full text Add to dashboard Cite

Cone-Beam CT (CBCT) is used in Interventional Radiology (IR) for identification of complex vascular anatomy, difficult to visualize in 2D fluoroscopy. However, long acquisition time makes CBCT susceptible to soft-tissue deformable motion that degrades visibility of fine vessels. We propose a targeted framework to compensate for deformable intra-scan motion via learned full-sequence models for identification of vascular anatomy coupled to an autofocus function specifically tailored to vascular imaging. The vessel-targeted autofocus acts in two stages: (i) identification of vascular and catheter targets in the projection domain; and (ii) autofocus optimization for a 4D vector field through an objective function that quantifies vascular visibility. Target identification is based on a deep learning model that operates on the complete sequence of projections, via a transformer encoder-decoder architecture that uses spatial-temporal self-attention modules to infer long-range feature correlations, enabling identification of vascular anatomy with highly variable conspicuity. The vascular autofocus function is derived through eigenvalues of the local image Hessian, which quantify the local image structure for identification of bright tubular structures. Motion compensation was achieved via spatial transformer operators that impart time dependent deformations to NPAR = 90 partial angle reconstructions, allowing for efficient minimization via gradient backpropagation. The framework was trained and evaluated in synthetic abdominal CBCTs obtained from liver MDCT volumes and including realistic models of contrast-enhanced vascularity with 15 to 30 end branches, 1 – 3.5 mm vessel diameter, and 1400 HU contrast. The targeted autofocus resulted in qualitative and quantitative improvement in vascular visibility in both simulated and clinical intra-procedural CBCT. The transformer-based target identification module resulted in superior detection of target vascularity and a lower number of false positives, compared to a baseline U-Net model acting on individual projection views, reflected as a 1.97x improvement in intersection-over-union values. Motion compensation in simulated data yielded improved conspicuity of vascular anatomy, and reduced streak artifacts and blurring around vessels, as well as recovery of shape distortion. These improvements amounted to an average 147% improvement in cross correlation computed against the motion-free ground truth, relative to the un-compensated reconstruction. Targeted autofocus yielded improved visibility of vascular anatomy in abdominal CBCT, providing better potential for intra-procedural tracking of fine vascular anatomy in 3D images. The proposed method poses an efficient solution to motion compensation in task-specific imaging, with future application to a wider range of imaging scenarios.

show abstract

“…This often results in artifacts from patient motion, which, in the case of abdominal imaging, is caused by a combination of periodic and aperiodic sources into a complex deformable motion field with non-predictable, highly heterogeneous, temporal trajectories. Previous work [3,4] demonstrated the potential of image-based autofocus for motion compensation in interventional CBCT for non-periodic motion, via data-driven autofocus metrics based on deep convolutional neural networks (CNNs). Deep autofocus metrics were trained to reproduce measures of structural similarity (Visual Information Fidelity -VIF-in our case), removing the need for a motion-free reference, yielding a learned metric (DL-VIF) that effectively quantified the degree of motion contamination and shape distortion, as well as the anatomical realism of the image content.…”

Section: Introductionmentioning

confidence: 99%

“…Deep autofocus yielded robust performance in head rigid motion compensation [3]. Extension to deformable motion was achieved by incorporating context information into DL-VIF, with a context-aware deep CNN design, integrated into in a multi-region autofocus framework [4].…”

Section: Introductionmentioning

confidence: 99%

Multi-stage Adaptive Sampling Autofocus (MASA) with a learned metric for deformable motion compensation in interventional cone-beam CT

Huang

Siewerdsen²,

et al. 2023

Medical Imaging 2023: Physics of Medical Imaging

View full text Add to dashboard Cite

Cone-beam CT (CBCT) is widespread in abdominal interventional imaging, but its long acquisition time makes it susceptible to patient motion. Image-based autofocus has shown success in CBCT deformable motion compensation, via deep autofocus metrics and multi-region optimization, but it is challenged by the large parameter dimensionality required to capture intricate motion trajectories. This work leverages the differentiable nature of deep autofocus metrics to build a novel optimization strategy, Multi-Stage Adaptive Spine Autofocus (MASA), for compensation of complex deformable motion in abdominal CBCT. MASA poses the autofocus problem as a multi-stage adaptive sampling strategy of the motion trajectory, sampled with Hermite spline basis with variable amplitude and knot temporal positioning. The adaptive method permits simultaneous optimization of the sampling phase, local temporal sampling density, and time-dependent amplitude of the motion trajectory. The optimization is performed in a multi-stage schedule with increasing number of knots that progressively accommodates complex trajectories in late stages, preconditioned by coarser components from early stages, and with minimal increase in dimensionality. MASA was evaluated in controlled simulation experiments with two types of motion trajectories: i) combinations of slow drifts with sudden jerk (sigmoid) motion; and ii) combinations of periodic motion sources of varying frequency into multi-frequency trajectories. Further validation was obtained in clinical data from liver CBCT featuring motion of contrast-enhanced vessels, and soft-tissue structures. The adaptive sampling strategy provided successful motion compensation in sigmoid trajectories, compared to fixed sampling strategies (mean SSIM increase of 0.026 compared to 0.011). Inspection of the estimated motion showed the capability of MASA to automatically allocate larger sampling density to parts of the scan timeline featuring sudden motion, effectively accommodating complex motion without increasing the problem dimension. Experiments on multifrequency trajectories with 3-stage MASA (5, 10, and 15 knots) yielded a twofold SSIM increase compared to single-stage autofocus with 15 knots (0.076 vs 0.040, respectively). Application of MASA to clinical datasets resulted in simultaneous improvement on the delineation of both contrast-enhanced vessels and soft-tissue structures in the liver. A new autofocus framework, MASA, was developed including a novel multi-stage technique for adaptive temporal sampling of the motion trajectory in combination with fully differentiable deep autofocus metrics. This novel adaptive sampling approach is a crucial step for application of deformable motion compensation to complex temporal motion trajectories.

show abstract

Context-aware, reference-free local motion metric for CBCT deformable motion compensation

Cited by 4 publications

References 10 publications

Deformable motion compensation in interventional cone‐beam CT with a context‐aware learned autofocus metric

Deformable motion compensation in interventional cone‐beam CT with a context‐aware learned autofocus metric

Deformable motion compensation for intraprocedural vascular cone-beam CT with sequential projection domain targeting and vessel-enhancing autofocus

Multi-stage Adaptive Sampling Autofocus (MASA) with a learned metric for deformable motion compensation in interventional cone-beam CT

Contact Info

Product

Resources

About