Multiple Landmark Detection Using Multi-agent Reinforcement Learning

Abstract

The detection of anatomical landmarks is a vital step for medical image analysis and applications for diagnosis, interpretation and guidance. Manual annotation of landmarks is a tedious process that requires domain-specific expertise and introduces inter-observer variability. This paper proposes a new detection approach for multiple landmarks based on multi-agent reinforcement learning. Our hypothesis is that the position of all anatomical landmarks is interdependent and nonrandom within the human anatomy, thus finding one landmark can help to deduce the location of others. Using a Deep Q-Network (DQN) architecture we construct an environment and agent with implicit intercommunication such that we can accommodate K agents acting and learning simultaneously, while they attempt to detect K different landmarks. During training the agents collaborate by sharing their accumulated knowledge for a collective gain. We compare our approach with state-of-the-art architectures and achieve significantly better accuracy by reducing the detection error by 50%, while requiring fewer computational resources and time to train compared to the naïve approach of training K agents separately. Code and visualizations available: https://github. com/thanosvlo/MARL-for-Anatomical-Landmark-Detection

DeepNavNet: Automated Landmark Localization for Neuronavigation

“…Even so, establishing an effective decision space and award function that reliably converges is non-trivial for DRL approaches in dynamic environments, and is especially a barrier for using such agents where consistently accurate and safe results are required (Dulac-Arnold et al, 2019). A multi-agent collaborative DRL approach, introduced in 2019, achieved an average localization error of 0.93 ± 0.18 and 1.05 ± 0.25 mm for AC and PC points, respectively (Vlontzos et al, 2019). Although our current supervised deep learning approach is more accurate, there is an acceleration of innovation in DRL methods and such artificial agents are expected to improve through experience as they navigate through more neuroimaging data, such as our ACPC-MRI -1 and ACPC-MRI-2 datasets.…”

Reinforced Redetection of Landmark in Pre- and Post-operative Brain Scan Using Anatomical Guidance for Image Alignment

“…Additionally, they evaluate several Q-Learning approaches as there are Double, Dueling and Dueling Double Q-Learning. A variant of their approach towards automatic view planning is shown in [1] and an extension towards the detection of multiple landmarks simultaneously in [15].…”

“…The recent success of RL in the field of landmark localization [2,[5][6][7]15] in combination with the ability of RL agents to adapt to a specific environment, encouraged us to transfer this approach for landmark redetection to pre-and post-operative brain images. Furthermore, RL has the benefit of being able to perform on limited training data, which is crucial for our task.…”

Enhanced Detection of Fetal Pose in 3D MRI by Deep Reinforcement Learning with Physical Structure Priors on Anatomy

“…A multi-scale strategy [6] and hierarchical action steps [1] have been proposed to further improve the performance of this approach. For the detection of multiple landmarks, [17] proposed a collaborative DQN based on concurrent Partially Observable Markov Decision Process [7]. However, the physical structure and communications among agents are not taken into consideration during decision making.…”

“…Moreover, most landmark-searching DRL only adopt an immediate reward based on the distance between the agent and its target landmark [1,5,6,17]. The smallest and most peripheral joints, i.e.…”