Prostate Motion Modelling Using Biomechanically-Trained Deep Neural Networks on Unstructured Nodes

Abstract

In this paper, we propose to train deep neural networks with biomechanical simulations, to predict the prostate motion encountered during ultrasound-guided interventions. In this application, unstructured points are sampled from segmented pre-operative MR images to represent the anatomical regions of interest. The point sets are then assigned with point-specific material properties and displacement loads, forming the un-ordered input feature vectors. An adapted PointNet can be trained to predict the nodal displacements, using finite element (FE) simulations as ground-truth data. Furthermore, a versatile bootstrap aggregating mechanism is validated to accommodate the variable number of feature vectors due to different patient geometries, comprised of a training-time bootstrap sampling and a model averaging inference. This results in a fast and accurate approximation to the FE solutions without requiring subject-specific solid meshing. Based on 160,000 nonlinear FE simulations on clinical imaging data from 320 patients, we demonstrate that the trained networks generalise to unstructured point sets sampled directly from holdout patient segmentation, yielding a near real-time inference and an expected error of 0.017 mm in predicted nodal displacement.

A review of artificial intelligence in prostate cancer detection on imaging

“…In an attempt to improve registration between the two modalities, several studies used pre-defined corresponding anatomical structures. [156][157][158][159]161,162 Some approaches used deformable transformations 161 to model patient movement, surrounding organs, for example, bladder and rectum, or interaction with surgical instrument, for example, biopsy needles and ultrasound probes. Others used AI models without constrained transformation models, [156][157][158] or prior knowledge in modeling soft tissue motion.…”

“…Others used AI models without constrained transformation models, [156][157][158] or prior knowledge in modeling soft tissue motion. 163,164 AI models have also been proposed to learn similarity measures 165 or transformation models from either biomechanical simulations (which emphasize biologically meaningful registration) 159 or shape populations. 166 A popular class of methods utilize prostate gland segmentations on both MR and ultrasound images before registering the resulting point sets 162 (Figure 4).…”

Real-time multimodal image registration with partial intraoperative point-set data

“…This will provide a loss computed from, in general, stronger supervision and , while test data at inference are more likely to have a different distribution that is similar to what is represented by and . Training-time bootstrap resampling ( Saeed et al., 2020 ), when , , or is large or the difference between their sizes – i.e., the difference in the number of points – is large. This allows sampling a subset of any of these point-sets during a stochastic or mini-batch gradient descent while maintaining an unbiased gradient.…”

“…Training-time bootstrap resampling ( Saeed et al., 2020 ), when , , or is large or the difference between their sizes – i.e., the difference in the number of points – is large. This allows sampling a subset of any of these point-sets during a stochastic or mini-batch gradient descent while maintaining an unbiased gradient.…”

Intra-operative Update of Boundary Conditions for Patient-Specific Surgical Simulation

“…A recent research trend has focused on the usage of Deep Neural Networks (DNNs) to update a biomechanical model based on intra-operative data [10,4,17,16,19]. These works have shown that DNNs can learn biomechanical models even when trained with synthetic data only, while guaranteeing very low inference time.…”