Development and Validation of a Novel Methodological Pipeline to Integrate Neuroimaging and Photogrammetry for Immersive 3D Cadaveric Neurosurgical Simulation

Abstract

BackgroundVisualizing and comprehending 3-dimensional (3D) neuroanatomy is challenging. Cadaver dissection is limited by low availability, high cost, and the need for specialized facilities. New technologies, including 3D rendering of neuroimaging, 3D pictures, and 3D videos, are filling this gap and facilitating learning, but they also have limitations. This proof-of-concept study explored the feasibility of combining the spatial accuracy of 3D reconstructed neuroimaging data with realistic texture and fine anatomical details from 3D photogrammetry to create high-fidelity cadaveric neurosurgical simulations.MethodsFour fixed and injected cadaver heads underwent neuroimaging. To create 3D virtual models, surfaces were rendered using magnetic resonance imaging (MRI) and computed tomography (CT) scans, and segmented anatomical structures were created. A stepwise pterional craniotomy procedure was performed with synchronous neuronavigation and photogrammetry data collection. All points acquired in 3D navigational space were imported and registered in a 3D virtual model space. A novel machine learning-assisted monocular-depth estimation tool was used to create 3D reconstructions of 2-dimensional (2D) photographs. Depth maps were converted into 3D mesh geometry, which was merged with the 3D virtual model’s brain surface anatomy to test its accuracy. Quantitative measurements were used to validate the spatial accuracy of 3D reconstructions of different techniques.ResultsSuccessful multilayered 3D virtual models were created using volumetric neuroimaging data. The monocular-depth estimation technique created qualitatively accurate 3D representations of photographs. When 2 models were merged, 63% of surface maps were perfectly matched (mean [SD] deviation 0.7 ± 1.9 mm; range −7 to 7 mm). Maximal distortions were observed at the epicenter and toward the edges of the imaged surfaces. Virtual 3D models provided accurate virtual measurements (margin of error <1.5 mm) as validated by cross-measurements performed in a real-world setting.ConclusionThe novel technique of co-registering neuroimaging and photogrammetry-based 3D models can (1) substantially supplement anatomical knowledge by adding detail and texture to 3D virtual models, (2) meaningfully improve the spatial accuracy of 3D photogrammetry, (3) allow for accurate quantitative measurements without the need for actual dissection, (4) digitalize the complete surface anatomy of a cadaver, and (5) be used in realistic surgical simulations to improve neurosurgical education.

Virtual neurosurgery anatomy laboratory: A collaborative and remote education experience in the metaverse

“…We used the immersive anatomy model environments developed with various forms of photogrammetry, including a novel technique based on the use of machine learning for the 3D rendering of monoscopic images (i.e., monoscopic photogrammetry). [ 12 , 14 ] This technique allows the incorporation of anatomical features, including textures, lighting, and colors, into the model, which provides improved visualization of microanatomical structures that cannot be fully appreciated with other methods, such as reconstructions based on Digital Imaging and Communications in Medicine. [ 25 ] In addition, the use of a VR headset with head tracking allowed depth discrimination of anatomical structures and easy exploration of different neurosurgical perspectives.…”

“…The end-product is navigable and volumetric surface reconstruction can be imported to extended-reality interfaces. [ 12 , 14 ]…”

“…A database of volumetric anatomy renderings created from cadaveric dissections was developed using photogrammetry, a technique that allows 3D models to be created from highresolution photographs. [8,13,14] e simulation experience was enhanced with 3D renderings of surgical equipment were not used during the training process. Depth estimations are used for 3D reprojection and volumetric point cloud creation from a single anatomical image.…”

“…e end-product is navigable and volumetric surface reconstruction can be imported to extended-reality interfaces. [12,14] All 3D reconstructions were performed using a laptop (MacBook Pro, Apple, Inc.) with a 2.5-GHz quad-core Intel Core i7 processor and a dedicated graphics card (NVIDIA GeForce GT 750M; Nvidia Corp., Santa Clara, CA). e 3D scanning was performed using the Qlone app (Qlone 2017-2022, EyeCue Vision Technologies, Ltd.,) on an 11-inch second-generation iPad Pro (Apple, Inc.).…”

Anatomic Depth Estimation and Three-Dimensional Reconstruction of Microsurgical Anatomy Using Monoscopic High-Definition Photogrammetry and Machine Learning

“…We established that machine learning is capable of delivering plausible anatomic depth estimations for specific sets of ROIs using only 1 image as input. 4 This does not mean that depth allocations will be always accurate; we believe it is necessary to perform a cross-check against neuronavigation or Digital Imaging and Communications in Medicine data to verify the accuracy of estimations. We did observe, however, that in the case of Rhoton Collection models, even without ground truth measurements available, overall 3D reconstruction was possible and anatomically consistent.…”