An Unsupervised Convolutional LSTM Network (C-LSTMNet) for Lung 4D-CT Registration

Abstract

This study proposed a novel method for lung Four-Dimensional Computed Tomography (4D-CT) deformable image registration (DIR) called convolutional long-short term memory network (C-LSTMNet). It allowed us to capture temporal and spatial features between current and previous phases, as well as perform groupwise registration for multiple image pairs in 4D-CT datasets. The proposed registration framework utilized a recurrent neural network (RNN) to accurately predict displacement vector fields (DVFs) between multiple image pairs in an end-to-end manner, taking into account the spatiotemporal relationship between phase images. The input consisted of a sequence of Three-Dimensional Computed Tomography (3D-CT) images captured from inspiratory phase to expiratory phase within a complete breathing cycle. In this sequence, the first phase image was defined as the target image, while the other phase images were considered as moving images. Multiple C-LSTM units were stacked together to capture temporal clues between these images. The proposed C-LSTMNet was trained using 85 collected 4D-CT datasets from lung cancer patients without supervision, and its efficiency was evaluated by employing an open-source 4D-CT dataset from dirlab. Target registration error (TRE) was measured and compared between C-LSTMNet and 4 recently published registration methods in a control group. The mean and standard deviation of TRE in C-LSTMNet were 1.30±0.87 mm, which outperformed other existing deep-learning methods in the control group in this study, and the calculation time for each forward prediction was about 0.45 seconds. The preliminary results on oncologic patients demonstrated that the proposed C-LSTMNet had the potential to accurately and quickly synchronize lung 4D-CT.INDEX TERMS 4D-CT, recurrent neural network, groupwise registration, long-short term memory network, target registration error.