Region Proposal Network with Graph Prior and Iou-Balance Loss for Landmark Detection in 3D Ultrasound

Abstract

3D ultrasound (US) can facilitate detailed prenatal examinations for fetal growth monitoring. To analyze a 3D US volume, it is fundamental to identify anatomical landmarks of the evaluated organs accurately. Typical deep learning methods usually regress the coordinates directly or involve heatmapmatching. However, these methods struggle to deal with volumes with large sizes and the highly-varying positions and orientations of fetuses. In this work, we exploit an object detection framework to detect landmarks in 3D fetal facial US volumes. By regressing multiple parameters of the landmarkcentered bounding box (B-box) with a strict criteria, the proposed model is able to pinpoint the exact location of the targeted landmarks. Specifically, the model uses a 3D region proposal network (RPN) to generate 3D candidate regions, followed by several 3D classification branches to select the best candidate. It also adopts an IoU-balance loss to improve communications between branches that benefits the learning process. Furthermore, it leverages a distance-based graph prior to regularize the training and helps to reduce false positive predictions. The performance of the proposed framework is evaluated on a 3D US dataset to detect five key fetal facial landmarks. Results showed the proposed method outperforms some of the state-of-the-art methods in efficacy and efficiency.

Toward deep observation: A systematic survey on artificial intelligence techniques to monitor fetus via ultrasound images

“…In ( Singh et al., 2021a ), 3D US images taken in the second and third trimesters were segmented to identify background, face mask (excluding facial structures), eyes, nose, and lips. In another study the object detection method was used on 3D US images in ( Chen et al., 2020c ) to detect the left fetal eye, middle eyebrow, right eye, nose, and chin. Furthermore, classification used 2D US images taken in the first and second trimesters to detect the nasal bone.…”

“…Of these studies (n = 10, 9.34%), we did not find any public dataset available online. However, within the private dataset, image augmentation was applied in (n = 3, 2.8%) studies ( Chen et al., 2020c ; Miyagi et al., 2021 ; Singh et al., 2021a ). The k-fold cross-validation was employed in ( Lei et al., 2014 ; Singh et al., 2021a ; Wang et al., 2021 ).…”

“…DL was used heavily in all fetal organ experiments including general fetus issues, fetus head, fetus face, fetus heart, and fetus abdomen. Furthermore, as seen in most of the models (n = 74, 69.1%), Convolutional Neural Network (CNN) used as a backbone of deep learning and is used in all tasks including classification, segmentation, and object detection ( Aji et al., 2019 ; Al-Bander et al., 2020 ; Andriani and Mardhiyah, 2019 ; Arnaout et al., 2021 ; Brahma et al., 2021 ; Budd et al., 2019 ; Cai et al., 2020 , 2018 ; Chen et al., 2014 , 2017 , 2020a , 2020b , 2020c ; Deepika et al., 2021 ; Desai et al., 2020 ; Dinesh Simon and Kavitha, 2021 ; Dong et al., 2020 ; Dozen et al., 2020 ; Droste et al., 2020 ; Fathimuthu Joharah and Mohideen, 2020 ; Fiorentino et al., 2021 ; Gao and Noble, 2019 ; Gong et al., 2020 ; Huang et al., 2018 ; Jang et al., 2017 ; Kim et al., 2018 , 2019a ; Komatsu et al., 2021b ; Li et al., 2020 ; Li et al., 2017 ; Lin et al., 2019a , 2019 ; Liu et al., 2020 ; Liu et al., 2019 ; Miyagi et al., 2021 ; Namburete and Noble, 2013 ; Nie et al., 2015a ; Patra et al., 2017 ; Perez-Gonzalez et al., 2020 ; Prieto et al., 2021 ; Pu et al., 2021 ; Qiao and Zulkernine, 2020 ; Qu et al., 2020a ; Ravishankar et al., 2016 ; Ryou et al., 2016 , 2019 ; Sahli et al., 2020 ; Singh et al., 2021a , 2021b ; Sobhaninia et al., 2019 , …”

Effective Face Detector Based on YOLOv5 and Superresolution Reconstruction

“…In most anchor-based [ 34 ] methods, it is used not only to judge the positive and negative sample but also to assess the distance between the location of the predicted box and the ground truth. The paper proposes that a regression positioning loss [ 35 ] should be considered: overlapping area, center point distance, and aspect ratio, which have aroused wide concern. At present, more and more researchers propose better performance algorithms, such as IOU, GIOU, DIOU, and CIOU.…”

A Review on Deep-Learning Algorithms for Fetal Ultrasound-Image Analysis

“…The model obtains a mean ED of 1.72 mm over a five-fold cross-validation with 6 volumes for testing in each fold. In [162], an RPN-based object detection framework is proposed to detect landmarks in 3D facial US volumes. Predictions from the RPN architecture are further refined with a distance-based graph prior to produce the final bounding box for each landmark.…”