Enhanced Detection of Fetal Pose in 3D MRI by Deep Reinforcement Learning with Physical Structure Priors on Anatomy

Abstract

Fetal MRI is heavily constrained by unpredictable and substantial fetal motion that causes image artifacts and limits the set of viable diagnostic image contrasts. Current mitigation of motion artifacts is predominantly performed by fast, single-shot MRI and retrospective motion correction. Estimation of fetal pose in real time during MRI stands to benefit prospective methods to detect and mitigate fetal motion artifacts where inferred fetal motion is combined with online slice prescription with low-latency decision making. Current developments of deep reinforcement learning (DRL), offer a novel approach for fetal landmarks detection. In this task 15 agents are deployed to detect 15 landmarks simultaneously by DRL. The optimization is challenging, and here we propose an improved DRL that incorporates priors on physical structure of the fetal body. First, we use graph communication layers to improve the communication among agents based on a graph where each node represents a fetal-body landmark. Further, additional reward based on the distance between agents and physical structures such as the fetal limbs is used to fully exploit physical structure. Evaluation of this method on a repository of 3-mm resolution in vivo data demonstrates a mean accuracy of landmark estimation within 10 mm of ground truth as 87.3%, and a mean error of 6.9 mm. The proposed DRL for fetal pose landmark search demonstrates a potential clinical utility for online detection of fetal motion that guides real-time mitigation of motion artifacts as well as health diagnosis during MRI of the pregnant mother.

AFFIRM: Affinity Fusion-based Framework for Iteratively Random Motion correction of multi-slice fetal brain MRI

“…Salehi et al trained regression CNNs to estimate the 3D positions of the fetal brain using individual slices and also provided a fast SVR strategy [31]. Zhang et al employed an end-to-end and multi-agent deep reinforcement learning network to detect the landmarks of the fetal pose in each time frame [32].…”

Stochastic‐offset‐enhanced restricted slice excitation and 180° refocusing designs with spatially non‐linear ΔB0 shim array fields

“…4 To further improve motion robustness, reduced FOV (rFOV) fetal brain imaging could substantially reduce acquisition time because the region of interest (ROI), the fetal brain slice, occupies only a small fraction of the gravid abdomen and requires many fewer phase-encoding to fully sample. With fast fetal brain tracking and pose estimation techniques, 5,6 2D spatially selective pulses could excite a slice restricted to the fetal brain in real time.…”

Deep Reinforcement Learning Method for 3D-CT Nasopharyngeal Cancer Localization with Prior Knowledge

“…Experiments have shown that, given a series of annotated images, the agent can learn the best path to converge to the marked position. In response to the issue of artifacts in 3D fetal MRI images, Zhang et al [13] deployed 15 agents based on DRL, simultaneously detecting 15 feature markers, and set additional rewards based on the distance between the agent and the fetal body nodes to improve detection accuracy. Pesce et al [14] studied how to locate lung lesions on chest radiographs.…”