Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans

Abstract

Robust and fast detection of anatomical structures is a prerequisite for medical image analysis. Current solutions for anatomy detection are typically based on machine learning and are subject to several limitations, including the use of suboptimal feature engineering techniques and most importantly the use of computationally suboptimal search-schemes. To address these issues, we propose a method that follows a new paradigm by reformulating the detection problem as a behavior learning task for an artificial agent. We couple the modeling of the anatomy appearance and the object search in a unified behavioral framework, using the capabilities of deep reinforcement learning and multi-scale image analysis. In other words, an artificial agent is trained not only to distinguish the target anatomical object from the rest of the body but also how to find the object by learning and following an optimal navigation path to the target object in the imaged volumetric space. We evaluate our approach on 1487 3D-CT volumes from 532 patients and show that we significantly outperform state-of-the-art solutions on detecting several anatomical structures with no failed cases, while also improving the detection accuracy by 20-30%. Most importantly, we improve the detection-speed of the reference methods by 2-3 orders of magnitude, achieving unmatched real-time performance on large 3D-CT scans.

A gentle introduction to deep learning in medical image processing

“…In terms of structure and organization, we follow [22] here, but add recent developments in physical simulation and image reconstruction. [71] and the X-ray transform-invariant landmark detection by Bier et al [67] (projection image courtesy of Dr. Unberath). The right hand side shows a U-net-based stent segmentation after Breininger et al [72].…”

Multiple Landmark Detection Using Multi-agent Reinforcement Learning

“…During testing, we terminate the episode when the agent starts to oscillate around a position or exceeds a defined maximum number of frames seen in the episode similar to [2]. Collaborative Agents Previous approaches to the problem of landmark detection by [2], [7] and [8] considered a single agent looking for a single landmark. This means that further landmarks needs to be trained with separate instances of the agent making a large scale application unfeasible.…”

“…The artificial agent learns to search and detect landmarks efficiently in an RL scenario. This search can be performed using fixed or multi-scale step strategies [7]. Recently, Alansary et al [2] proposed the use of different Deep Q-Network (DQN) architectures for landmark detection with novel hierarchical action steps.…”

Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents

“…Recently, DQN has shown promising results when employed in related applications in the medical imaging domain. Ghesu et al [3] introduced an automatic landmark detection approach using a DQN-agent to navigate in 3D images with fixed step actions. Maicas et al [7] proposed a similar method for breast lesion detection using actions to control the location and size of the bounding box.…”