In this paper we present an approach combining a finite element method and a deep neural network to learn complex elastic deformations with the objective of providing augmented reality during hepatic surgery. Derived from the U-Net architecture, our network is built entirely from physically-based simulations of a preoperative segmentation of the organ. These simulations are performed using an immersedboundary method, which offers several numerical and practical benefits, such as not requiring boundary-conforming volume elements. We perform a quantitative assessment of the method using synthetic and ex vivo patient data. Results show that the network is capable of solving the deformed state of the organ using only a sparse partial surface displacement data and achieve similar accuracy as a FEM solution, while being about 100x faster. When applied to an ex vivo liver example, we achieve the registration in only 3 ms with a mean target registration error (TRE) of 2.9 mm.

In the context of ultrasound (US) guided breast biopsy, image fusion techniques can be employed to track the position of USinvisible lesions previously identified on a pre-operative image. Such methods have to account for the large anatomical deformations resulting from probe pressure during US scanning within the real-time constraint. Although biomechanical models based on the finite element (FE) method represent the preferred approach to model breast behavior, they cannot achieve real-time performances. In this paper we propose to use deep neural networks to learn large deformations occurring in ultrasoundguided breast biopsy and then to provide accurate prediction of lesion displacement in real-time. We train a U-Net architecture on a relatively small amount of synthetic data generated in an offline phase from FE simulations of probe-induced deformations on the breast anatomy of interest. Overall, both training data generation and network training are performed in less than 5 hours, which is clinically acceptable considering that the biopsy can be performed at most the day after the pre-operative scan. The method is tested both on synthetic and on real data acquired on a realistic breast phantom. Results show that our method correctly learns the deformable behavior modelled via FE simulations and is able to generalize to real data, achieving a target registration error comparable to that of FE models, while being about a hundred times faster.

This paper proposes a fast, stable and accurate meshless method to simulate geometrically non-linear elastic behaviors. To address the inherent limitations of finite element (FE) models, the discretization of the domain is simplified by removing the need to create polyhedral elements. The volumetric locking effect exhibited by incompressible materials in some linear FE models is also completely avoided. Our approach merely requires that the volume of the object be filled with a cloud of points. To minimize numerical errors, we construct a corotational formulation around the quadrature positions that is well suited for large displacements containing small deformations. The equations of motion are integrated in time following an implicit scheme. The convergence rate and accuracy are validated through both stretching and bending case studies. Finally, results are presented using a set of examples that show how we can easily build a realistic physical model of various deformable bodies with little effort spent on the discretization of the domain.

No abstract

No abstract