Pancreatic Tumor Growth Prediction With Elastic-Growth Decomposition, Image-Derived Motion, and FDM-FEM Coupling

Wong, Ken C. L.; Summers, Ronald M.; Kebebew, Electron; Yao, Jianhua

doi:10.1109/tmi.2016.2597313

Cited by 48 publications

(79 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…sequence data set for training and testing. We acknowledge that there might be some bias of using simple tumor centroids for the longitudinal alignment, but this is a relatively (more) reliable approach compared to the image appearance based registration methods, based on our preliminary experiment and past studies (e.g., [8], [12], [13]).…”

Section: St-clstm (St-1)mentioning

confidence: 99%

Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data

Zhang

Lü

Wang

et al. 2020

IEEE Trans. Med. Imaging

Self Cite

View full text Add to dashboard Cite

Prognostic tumor growth modeling via volumetric medical imaging observations can potentially lead to better outcomes of tumor treatment management and surgical planning. Recent advances of convolutional networks (ConvNets) have demonstrated higher accuracy than traditional mathematical models can be achieved in predicting future tumor volumes. This indicates that deep learning based data-driven techniques may have great potentials on addressing such problem. However, current 2D image patch based modeling approaches can not make full use of the spatio-temporal imaging context of the tumor's longitudinal 4D (3D + time) patient data. Moreover, they are incapable to predict clinically-relevant tumor properties, other than the tumor volumes. In this paper, we exploit to formulate the tumor growth process through convolutional Long Short-Term Memory (ConvLSTM) that extract tumor's static imaging appearances and simultaneously capture its temporal dynamic changes within a single network. We extend ConvLSTM into the spatio-temporal domain (ST-ConvLSTM) by jointly learning the inter-slice 3D contexts and the longitudinal or temporal dynamics from multiple patient studies. Our approach can incorporate other nonimaging patient information in an end-to-end trainable manner. Experiments are conducted on the largest 4D longitudinal tumor dataset of 33 patients to date. Results validate that the proposed ST-ConvLSTM model produces a Dice score of 83.2%±5.1% and a RVD of 11.2%±10.8%, both statistically significantly outperforming (p <0.05) other compared methods of traditional linear model, ConvLSTM, and generative adversarial network (GAN) under the metric of predicting future tumor volumes. Additionally, our new method enables the prediction of both cell density and CT intensity numbers. Last, we demonstrate the generalizability of ST-ConvLSTM by employing it in 4D medical image segmentation task, which achieves an averaged Dice score of 86.3%±1.2% for left-ventricle segmentation in 4D ultrasound with 3 seconds per patient case.

show abstract

Section: St-clstm (St-1)mentioning

confidence: 99%

Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data

Zhang

Lü

Wang

et al. 2020

IEEE Trans. Med. Imaging

Self Cite

View full text Add to dashboard Cite

show abstract

“…There has only been little work on model-based image analysis in brain tumor imaging based on PDE-constrained optimization [47,70,99,110,111,125,127,135,165]; 12 the works in [47,70,99,110,111,125,165] consider adjoint based approaches for numerical optimization. Others use derivative-free optimization [40,110,114,127,133,139,206,207], finite-difference approximations to the gradient [101], or tackle the parameter estimation problem within a Bayesian framework [48,91,116,117,122,123,138,154] (see Rem. 1).…”

Section: Data Assimilation In Brain Tumor Imagingmentioning

confidence: 99%

PDE-constrained optimization in medical image analysis

et al. 2018

View full text Add to dashboard Cite

PDE-constrained optimization problems find many applications in medical image analysis, for example, neuroimaging, cardiovascular imaging, and oncological imaging. We review related literature and give examples on the formulation, discretization, and numerical solution of PDE-constrained optimization problems for medical imaging. We discuss three examples. The first one is image registration. The second one is data assimilation for brain tumor patients, and the third one data assimilation in cardiovascular imaging. The image registration problem is a classical task in medical image analysis and seeks to find pointwise correspondences between two or more images. The data assimilation problems use a PDE-constrained formulation to link a biophysical model to patient-specific data obtained from medical images. The associated optimality systems turn out to be sets of nonlinear, multicomponent PDEs that are challenging to solve in an efficient way.The ultimate goal of our work is the design of inversion methods that integrate complementary data, and rigorously follow mathematical and physical principles, in an attempt to support clinical decision making. This requires reliable, high-fidelity algorithms with a short time-to-solution. This task is complicated by model and data uncertainties, and by the fact that PDE-constrained optimization problems are ill-posed in nature, and in general yield high-dimensional, severely ill-conditioned systems after discretization. These features make regularization, effective preconditioners, and iterative solvers that, in many cases, have to be implemented on distributed-memory architectures to be practical, a prerequisite. We showcase state-of-the-art techniques in scientific computing to tackle these challenges.A. MANG, A. GHOLAMI, C. DAVATZIKOS, AND G. BIROS (hard constraints 1 ) to incorporate additional prior knowledge on the expected deformation map. One motivation for adding such constraints is that often, especially in longitudinal studies of the same patient, there is a real deformation by the realization of an actual physical phenomenon. A simple constraint, which nonetheless poses significant numerical challenges, is the incompressibility of tissue [128,129,131]. Examples for more complex biophysical constraints are brain tumor growth models [73,99,174,[210][211][212] or cardiac motion models [188].We will discuss all of these constraints either in the context of diffeomorphic image registration or in the more generic context of data assimilation in medical imaging. This brings us to the second problem we are addressing in this article: Data assimilation in brain tumor imaging [70,100,114,135,138] (see §3). The PDE constraint is, in its simplest form, a nonlinear parabolic differential equation. The inversion variables are, e.g., the initial condition, the growth rate of the tumor, or a diffusion coefficient that controls the net migration of cancerous cells within brain parenchyma. The regularization model is in our case an L 2 -penalty. The third problem is cardiac motion...

show abstract

“…Pancreatic neuroendocrine tumors are slow-growing, and usually are not treated until they reach a certain size. To choose between nonoperative or surgical treatments, and to better manage the treatment planning, it is crucial to accurately predict the patient-specific spatio-temporal progression of pancreatic tumors [9]. The prediction of tumor growth is a very challenging task.…”

Section: Introductionmentioning

confidence: 99%

“…The prediction of tumor growth is a very challenging task. It has long been viewed as a mathematical modeling problem [2,5,9]. Clinical imaging data provide non-invasive and in vivo measurements of the tumor over time at a macroscopic level.…”

Section: Introductionmentioning

confidence: 99%

“…For this reason, previous works on image-based tumor growth modeling are mainly based on the reaction-diffusion equations and on biomechanical models. Some previous tumor growth models [2,5,9] are derived from two or more longitudinal imaging studies of a specific patient over time. While they yield promising results, they fail to account for the population trend of tumor growth patterns and specific tumor clinical characteristics.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Personalized Pancreatic Tumor Growth Prediction via Group Learning

Zhang

Lü

Summers

et al. 2017

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Tumor growth prediction, a highly challenging task, has long been viewed as a mathematical modeling problem, where the tumor growth pattern is personalized based on imaging and clinical data of a target patient. Though mathematical models yield promising results, their prediction accuracy may be limited by the absence of population trend data and personalized clinical characteristics. In this paper, we propose a statistical group learning approach to predict the tumor growth pattern that incorporates both the population trend and personalized data, in order to discover high-level features from multimodal imaging data. A deep convolutional neural network approach is developed to model the voxel-wise spatio-temporal tumor progression. The deep features are combined with the time intervals and the clinical factors to feed a process of feature selection. Our predictive model is pretrained on a group data set and personalized on the target patient data to estimate the future spatio-temporal progression of the patient's tumor. Multimodal imaging data at multiple time points are used in the learning, personalization and inference stages. Our method achieves a Dice coefficient of 86.8% ± 3.6% and RVD of 7.9% ± 5.4% on a pancreatic tumor data set, outperforming the DSC of 84.4%±4.0% and RVD 13.9%±9.8% obtained by a previous state-of-the-art model-based method.

show abstract

Pancreatic Tumor Growth Prediction With Elastic-Growth Decomposition, Image-Derived Motion, and FDM-FEM Coupling

Cited by 48 publications

References 34 publications

Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data

Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data

PDE-constrained optimization in medical image analysis

Personalized Pancreatic Tumor Growth Prediction via Group Learning

Contact Info

Product

Resources

About