Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound

Abstract

Standard plane localization is crucial for ultrasound (US) diagnosis. In prenatal US, dozens of standard planes are manually acquired with a 2D probe. It is time-consuming and operator-dependent. In comparison, 3D US containing multiple standard planes in one shot has the inherent advantages of less user-dependency and more efficiency. However, manual plane localization in US volume is challenging due to the huge search space and large fetal posture variation. In this study, we propose a novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US. Our contribution is twofold. First, we equip the RL framework with a landmark-aware alignment module to provide warm start and strong spatial bounds for the agent actions, thus ensuring its effectiveness. Second, instead of passively and empirically terminating the agent inference, we propose a recurrent neural network based strategy for active termination of the agent's interaction procedure. This improves both the accuracy and efficiency of the localization system. Extensively validated on our in-house large dataset, our approach achieves the accuracy of 3.4mm/9.6 • and 2.7mm/9.1 • for the transcerebellar and transthalamic plane localization, respectively. Our proposed RL framework is general and has the potential to improve the efficiency and standardization of US scanning.

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

“…We can see that the early-stop strategy has impacted on our proposed LaOML method from the experiments. Previous approaches in medical image analysis [40] introduce the RNN to map between the Q-value sequence and optimal step in value-function-based deep Q-Network (DQN) [41]. Nevertheless, differently to the complicated DQN methods, we employ policy gradient methods [29] to the fundamental heatmap refinement problem, improving its accuracy and efficiency directly.…”

Label Efficient Localization of Fetal Brain Biometry Planes in Ultrasound Through Metric Learning

“…Related Works: Recently, deep learning based approaches have shown success in recognition of fetal brain planes [2], [3], [4], [5] and [6]. Baumgartner et al [2] use a variant of VGG-Net, called SonoNet, for classification of 13 standardised fetal biometry planes, including TC and TV planes in fetal brain which was improved by [3] by introducing an attention mechanism.…”

“…Gao et al [5] model multi-scale spatial-temporal attention for detecting and tracking fetal structures, including the fetal head, but different to [2] and [3], does not specifically look at standard fetal biometry planes. Other works, such as [4] and [6], localize fetal brain standard planes in 3D ultrasound. Y. Li et al [4] uses a CNN to regress a rigid transformation iteratively for localising TC and TV planes in 3D fetal ultrasound.…”

“…Y. Li et al [4] uses a CNN to regress a rigid transformation iteratively for localising TC and TV planes in 3D fetal ultrasound. Dou et al [6] propose to localize TT and TV planes in 3D fetal ultrasound with a reinforcement learning framework, which can progressively interact with the volumes and modify the search trajectory towards the target planes. Yaqub et al [7] consider automatic quality assurance of the TV plane when it has been acquired.…”

“…Two experienced sonographers annotated three frames only per video (TCD, HC and background) and scored them. For training, we choose the annotated frames having a quality score ( 6). This resulted in: 301 HC frames, 208 TCD frames, 120 Other (background) frames and 33,957 unlabelled frames from 441 training videos (147 subjects).…”