Discriminative Correlation Filter Network for Robust Landmark Tracking in Ultrasound Guided Intervention

Shen, Chengshuo; He, Jishuai; Huang, Yibin; Wu, Jian

doi:10.1007/978-3-030-32254-0_72

Cited by 14 publications

(17 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, correlation filter (CF)-based 18,19 trackers have achieved great success in visual tracking, even in 2D US landmarks tracking. 1 However, it is difficult to apply a CFbased tracker to 3D landmark tracking. Since CF is derived from 2D matrix, it is not convenient to apply directly on 3D data.…”

Section: Methodsmentioning

confidence: 99%

“…Based on our previous work experience, 1 FCN consists of the following parts: three ConvBlocks and one local response normalization layer (LRN). Every block consists of a convolutional layer, batch normalization layer and Rectified Linear Units.…”

Section: Methodsmentioning

confidence: 99%

“…The values of these two hyper-parameters were determined according to our previous work. 1 Our network was trained only on the ILSVRC dataset, while it is evaluated on the CLUST dataset. This can show the generalization of our network.…”

Section: A Implementation Detailsmentioning

confidence: 99%

“…This might cause damage to crucial tissues and increase complication rates. 1 Although potential approaches, such as respiratory gating technology 2 and anesthesia, 3 aim to mitigate the uncertainties from abdomen motion, they may cause additional trauma and increase the treatment time.…”

Section: Introductionmentioning

confidence: 99%

“…Abdominal interventional therapy requires the surgeon to add margins, such as additional ablation area, around the target due to abdomen motion caused by breathing, heartbeat, and body drift of patients. This might cause damage to crucial tissues and increase complication rates 1 . Although potential approaches, such as respiratory gating technology 2 and anesthesia, 3 aim to mitigate the uncertainties from abdomen motion, they may cause additional trauma and increase the treatment time.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Tracking 3D ultrasound anatomical landmarks via three orthogonal plane‐based scale discriminative correlation filter network

Huang

et al. 2021

Medical Physics

Self Cite

View full text Add to dashboard Cite

In abdominal interventional therapy, accurate motion tracking of the target is regarded as crucial to minimize trauma and optimize dosage delivery. Meanwhile, three-dimensional (3D) ultrasound (US) is an attractive modality to show the real-time motion pattern of the target. In this work, we developed an accurate and robust landmark tracking algorithm for 3D US sequences. Methods: The proposed algorithm introduces a three orthogonal planes (TOPs) based scale discriminative correlation filter network for 3D US landmarks tracking. First, we couple the fully convolutional network (FCN) with scale discriminative correlation filter (SDCF) to generate an effective tracker. And SDCF is reformulated as a differentiable layer, which ensures the network can perform scale learning and end-to-end training. Next, we train the end-to-end network on millions of natural images. Finally, we convert 3D US image to 2D three-channel image by TOP transformation and feed them to the proposed network for performing online tracking. Results: Online tracking performance was evaluated on the Challenge of Liver Ultrasound Tracking (CLUST) dataset with 22 sets of 3D US sequences, obtaining mean error of 1.63 AE 1.04 mm and 95th percentile (95%ile) error of 3.37 mm, when compared with manual annotations annotated by surgeons. Ablation study indicates that the promising results benefit from SDCF and scale learning, which alleviates the influence from deformation. The findings of the clinical analysis support that the proposed algorithm can work well with different initial patch sizes, which means that our algorithm has potential to lighten the burden of surgeons. Conclusions: We propose a flexible, accurate and robust landmark tracking algorithm for the imageguided interventions, and our algorithm is comparable with the state-of-the-art approaches. The tracking accuracy and robustness show that our algorithm has potential in 3D US-guided abdominal interventional therapies. Furthermore, more researches are needed to improve the computing speed of the algorithm to achieve real-time tracking

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: A Implementation Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Tracking 3D ultrasound anatomical landmarks via three orthogonal plane‐based scale discriminative correlation filter network

Huang

et al. 2021

Medical Physics

Self Cite

View full text Add to dashboard Cite

show abstract

Deep learning‐based motion tracking using ultrasound images

et al. 2021

View full text Add to dashboard Cite

Purpose Ultrasound (US) imaging is an established imaging modality capable of offering video‐rate volumetric images without ionizing radiation. It has the potential for intra‐fraction motion tracking in radiation therapy. In this study, a deep learning‐based method has been developed to tackle the challenges in motion tracking using US imaging. Methods We present a Markov‐like network, which is implemented via generative adversarial networks, to extract features from sequential US frames (one tracked frame followed by untracked frames) and thereby estimate a set of deformation vector fields (DVFs) through the registration of the tracked frame and the untracked frames. The positions of the landmarks in the untracked frames are finally determined by shifting landmarks in the tracked frame according to the estimated DVFs. The performance of the proposed method was evaluated on the testing dataset by calculating the tracking error (TE) between the predicted and ground truth landmarks on each frame. Results The proposed method was evaluated using the MICCAI CLUST 2015 dataset which was collected using seven US scanners with eight types of transducers and the Cardiac Acquisitions for Multi‐structure Ultrasound Segmentation (CAMUS) dataset which was acquired using GE Vivid E95 ultrasound scanners. The CLUST dataset contains 63 2D and 22 3D US image sequences respectively from 42 and 18 subjects, and the CAMUS dataset includes 2D US images from 450 patients. On CLUST dataset, our proposed method achieved a mean tracking error of 0.70 ± 0.38 mm for the 2D sequences and 1.71 ± 0.84 mm for the 3D sequences for those public available annotations. And on CAMUS dataset, a mean tracking error of 0.54 ± 1.24 mm for the landmarks in the left atrium was achieved. Conclusions A novel motion tracking algorithm using US images based on modern deep learning techniques has been demonstrated in this study. The proposed method can offer millimeter‐level tumor motion prediction in real time, which has the potential to be adopted into routine tumor motion management in radiation therapy.

show abstract

Temporal contexts for motion tracking in ultrasound sequences with information bottleneck

Sun

Huang

Zhang³

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundRecently, deep convolutional neural networks (CNNs) have been widely adopted for ultrasound sequence tracking and shown to perform satisfactorily. However, existing trackers ignore the rich temporal contexts that exists between consecutive frames, making it difficult for these trackers to perceive information about the motion of the target.PurposeIn this paper, we propose a sophisticated method to fully utilize temporal contexts for ultrasound sequences tracking with information bottleneck. This method determines the temporal contexts between consecutive frames to perform both feature extraction and similarity graph refinement, and information bottleneck is integrated into the feature refinement process.MethodsThe proposed tracker combined three models. First, online temporal adaptive convolutional neural network (TAdaCNN) is proposed to focus on feature extraction and enhance spatial features using temporal information. Second, information bottleneck (IB) is incorporated to achieve more accurate target tracking by maximally limiting the amount of information in the network and discarding irrelevant information. Finally, we propose temporal adaptive transformer (TA‐Trans) that efficiently encodes temporal knowledge by decoding it for similarity graph refinement. The tracker was trained on 2015 MICCAI Challenge on Liver Ultrasound Tracking (CLUST) dataset to evaluate the performance of the proposed method by calculating the tracking error (TE) between the predicted landmarks and the ground truth landmarks for each frame. The experimental results are compared with 13 state‐of‐the‐art methods, and ablation studies are conducted.ResultsOn CLUST 2015 dataset, our proposed model achieves a mean TE of 0.81 ± 0.74 mm and a maximum TE of 1.93 mm for 85 point‐landmarks across 39 ultrasound sequences in the 2D sequences. Tracking speed ranged from 41 to 63 frames per second (fps).ConclusionsThis study demonstrates a new integrated workflow for ultrasound sequences motion tracking. The results show that the model has excellent accuracy and robustness. Reliable and accurate motion estimation is provided for applications requiring real‐time motion estimation in the context of ultrasound‐guided radiation therapy.

show abstract

Discriminative Correlation Filter Network for Robust Landmark Tracking in Ultrasound Guided Intervention

Cited by 14 publications

References 7 publications

Tracking 3D ultrasound anatomical landmarks via three orthogonal plane‐based scale discriminative correlation filter network

Tracking 3D ultrasound anatomical landmarks via three orthogonal plane‐based scale discriminative correlation filter network

Deep learning‐based motion tracking using ultrasound images

Temporal contexts for motion tracking in ultrasound sequences with information bottleneck

Contact Info

Product

Resources

About