Purpose Veress needle (VN) insertion, if not correctly performed, could cause severe injuries to intra-abdominal organs and vessels. Therefore, cognitive and psychomotor skills training is needed. Virtual reality (VR) and haptic technologies have the potential to offer realistic simulations. Methods We developed a novel VR and haptic surgical simulator for VN insertion to teach trainees how to correctly puncture the abdominal wall, experiencing realistic tactile sensations throughout the simulation. The simulator allows for both procedural and realistic training. We released two different versions: the first using the OpenHaptics$$^{\text {TM}}$$ TM (OH) Toolkit and the second exploiting CHAI3D. We evaluated the learning effect using different performance indexes (time to perform the procedure, error in insertion angle, number of undesired contacts with organs) in an insertion task for both experienced urologists and students. Results A general improvement of the chosen performance indexes was registered in the second repetition of the task for both groups. From the questionnaires, the simulator leveraging OH provides the trainee with a more precise haptic feedback, whereas the one exploiting CHAI3D allows them to perform the procedure more easily thanks to the better visualization of the virtual environment. The results proved that the participants appreciated both implementations, and the System Usability Scale (SUS) test resulted in a “good” usability. Conclusion The haptics-based and VR simulator has shown the potential to be an important resource for the basic urological training in obtaining the pneumoperitoneum and improving the acquisition of the necessary psychomotor skills, allowing for extended and more effective training without compromising patient safety.

Purpose In obstetric ultrasound (US) scanning, the learner’s ability to mentally build a three-dimensional (3D) map of the fetus from a two-dimensional (2D) US image represents a major challenge in skill acquisition. We aim to build a US plane localisation system for 3D visualisation, training, and guidance without integrating additional sensors. Methods We propose a regression convolutional neural network (CNN) using image features to estimate the six-dimensional pose of arbitrarily oriented US planes relative to the fetal brain centre. The network was trained on synthetic images acquired from phantom 3D US volumes and fine-tuned on real scans. Training data was generated by slicing US volumes into imaging planes in Unity at random coordinates and more densely around the standard transventricular (TV) plane. Results With phantom data, the median errors are 0.90 mm/1.17$$^\circ $$ ∘  and 0.44 mm/1.21$$^\circ $$ ∘  for random planes and planes close to the TV one, respectively. With real data, using a different fetus with the same gestational age (GA), these errors are 11.84 mm/25.17$$^\circ $$ ∘ . The average inference time is 2.97 ms per plane. Conclusion The proposed network reliably localises US planes within the fetal brain in phantom data and successfully generalises pose regression for an unseen fetal brain from a similar GA as in training. Future development will expand the prediction to volumes of the whole fetus and assess its potential for vision-based, freehand US-assisted navigation when acquiring standard fetal planes.

Many tasks in robot-assisted surgery require planning and controlling manipulators' motions that interact with highly deformable objects. This study proposes a realistic, timebounded simulator based on Position-based Dynamics (PBD) simulation that mocks brain deformations due to catheter insertion for pre-operative path planning and intra-operative guidance in keyhole surgical procedures. It maximizes the probability of success by accounting for uncertainty in deformation models, noisy sensing, and unpredictable actuation. The PBD deformation parameters were initialized on a parallelepiped-shaped simulated phantom to obtain a reasonable starting guess for the brain white matter. They were calibrated by comparing the obtained displacements with deformation data for catheter insertion in a composite hydrogel phantom. Knowing the gray matter brain structures' different behaviors, the parameters were fine-tuned to obtain a generalized human brain model. The brain structures' average displacement was compared with values in the literature. The simulator's numerical model uses a novel approach with respect to the literature, and it has proved to be a close match with real brain deformations through validation using recorded deformation data of in-vivo animal trials with a mean mismatch of 4.73±2.15%. The stability, accuracy, and real-time performance make this model suitable for creating a dynamic environment for KN path planning, pre-operative path planning, and intra-operative guidance.

No abstract