Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging

Abstract

Computed Tomography (CT) takes X-ray measurements on the subjects to reconstruct tomographic images. As X-ray is radioactive, it is desirable to control the total amount of dose of X-ray for safety concerns. Therefore, we can only select a limited number of measurement angles and assign each of them limited amount of dose. Traditional methods such as compressed sensing usually randomly select the angles and equally distribute the allowed dose on them. In most CT reconstruction models, the emphasize is on designing effective image representations, while much less emphasize is on improving the scanning strategy. The simple scanning strategy of random angle selection and equal dose distribution performs well in general, but they may not be ideal for each individual subject. It is more desirable to design a personalized scanning strategy for each subject to obtain better reconstruction result. In this paper, we propose to use Reinforcement Learning (RL) to learn a personalized scanning policy to select the angles and the dose at each chosen angle for each individual subject. We first formulate the CT scanning process as an MDP, and then use modern deep RL methods to solve it. The learned personalized scanning strategy not only leads to better reconstruction results, but also shows strong generalization to be combined with different reconstruction algorithms. * Equal Contribution Preprint. Under review.

A Note on Machine Learning Approach for Computational Imaging

“…In this section, we discuss how we may use reinforcement learning (RL) [72], a powerful tool for sequential decision making, to improve sensing in computational imaging. In particular, we shall review the work of [64] for CT image reconstruction, where the we used RL to train a CT scanning policy that is adaptive to each imaging subject. Note that RL has also been used in other imaging modalities to improve sensing.…”

“…In [64], the scanning procedure is formulated as a Markov Decision Process (MDP), where the state includes currently collected measurements, the action determines the next measurement angle and the dose allocation, and the reward depends on the reconstruction quality measured by the peak signal-to-noise ratio (PSNR). Then, the scanning policy is trained using the proximal policy optimization (PPO) algorithm [62].…”

“…The key to the application of RL in sensing is the design of an appropriate MDP. Here, we review the MDP designed by [64] as follows.…”

“…4) The transition model represents the scanning process of CT. At time step t, given the state s t and action a t , the next state s t+1 is simply the concatenation of s t and s t+1 = (p t+1 , d ac t+1 , d rest t+1 ). Details on how each of the three elements in s t+1 is computed can be found in [64].…”

“…The RNN is also carefully designed to handle continuous action and discrete action simultaneously. We refer the interested readers to [64] for details. Numerical experiments on a large CT image dataset show that RL's scanning policy significantly outperforms the standard scanning strategies in terms of reconstruction quality.…”