Sub-millimeter precision 3D measurement through a standard endoscope with time of flight

Stolyarov, R. A.; Buharin, Vasiliy E.; Val, Michael; Beurskens, Charlie; DeMaio, Manuel; Srinivasan, Suraj; Calef, Tom; Kim, Peter

doi:10.1117/12.2605993

Cited by 2 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The usability of the system was tested in vivo on a pig abdomen, but no quantitative evaluation of the 3D reconstruction was reported [42]. Overall, structured-light technology has shown limited robustness and applicability to the clinical workflow [47]. As a more practical extension, Weld et al recently proposed a learning-based solution to stereo matching where disparity and structured light patterns are estimated jointly [48].…”

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

“…Stolyarov et al customized an off-the-shelf lidar to be coupled to a commercial endoscope [47]. To the best of our knowledge, this solution represents the most advanced research implementation of lidar sensing for endoscopy.…”

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

“…Still, the resolution is limited to 640 × 480 pixels, the proposed hardware setup is relatively cumbersome to build and use, and several steps of post-processing are necessary, such as spatial and temporal filtering as well as compensation for fiber optic distortion. Quantitative comparison with a laser scanner on ex-vivo porcine kidneys showed nearest neighbor average errors in the 0.75 mm to 10 mm range [47]. Interestingly, Stolyarov et al reported the need for testing ToF technology with alternative hardware solutions, at various working distances, and with different tissue targets, speculating that lidar measurement accuracy might depend on tissue type [47].…”

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

“…Quantitative comparison with a laser scanner on ex-vivo porcine kidneys showed nearest neighbor average errors in the 0.75 mm to 10 mm range [47]. Interestingly, Stolyarov et al reported the need for testing ToF technology with alternative hardware solutions, at various working distances, and with different tissue targets, speculating that lidar measurement accuracy might depend on tissue type [47].…”

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

See 3 more Smart Citations

Dense 3D Reconstruction Through Lidar: A New Perspective on Computer-Integrated Surgery

Caccianiga¹,

Kuchenbecker²

2022

Proceedings of the 14th Hamlyn Symposium on Medical Robotics 2022

View full text Add to dashboard Cite

Technical innovations in sensing and computation are quickly advancing the Ô¨Åeld of computer-integrated surgery. From one side, spectral imaging and biophoton- ics are strongly contributing to intraoperative diagnostics and decision making. Simultaneously, learning-based algorithms are reshaping the concept of assistance and prediction in surgery. In this fast-evolving panorama, we strongly believe there is still a need for robust geometric reconstruction of the surgical Ô¨Åeld whether the goal is traditional surgical assistance or partial or full autonomy. 3D reconstruction in surgery has been investigated almost only in the space of mono and stereoscopic visual imaging because surgeons always view the proce- dure through a clinical endoscope. Compared to using traditional computer vision, deep learning has made signiÔ¨Åcant progress in creating high-quality 3D recon- structions and dense maps from such data streams, especially for monocular simultaneous localization and mapping (SLAM) [1]. The main limitations are linked to reliability, generalization, and computational cost. Meanwhile, lidar (light detection and ranging) has greatly expanded in use, especially in SLAM for robotics, terrestrial vehicles, and drones. Lidar sensors explicitly measure the depth Ô¨Åeld rather than inferring it from camera images. The technology is evolving quickly thanks to the upsurge of mixed and augmented reality in consumer mobile devices [2]: high-resolution, short- range, miniaturized lidar sensors are expected soon. In parallel to these developments, the concept of multiple-viewpoint surgical imaging was proposed in the early 2010‚Äôs in the context of magnetic actuation and micro-invasive surgery [3]. In routine clinical practice, however, the use of multiple trans-abdominal cannulae still limits the kinematics of the camera and instruments to have a Ô¨Åxed pivot point at the body wall. For this reason, here we propose an approach in which each surgical cannula can potentially hold a miniature lidar. We envision that exploring this powerful sensing tech- nology and enabling multi-viewpoint imaging without disrupting the current surgical workÔ¨Çow will yield far more accurate and complete 3D reconstructions of the surgical Ô¨Åeld, which opens new opportunities for the future of computer-integrated surgery.

show abstract

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

See 2 more Smart Citations

Dense 3D Reconstruction Through Lidar: A New Perspective on Computer-Integrated Surgery

Caccianiga¹,

Kuchenbecker²

2022

Proceedings of the 14th Hamlyn Symposium on Medical Robotics 2022

View full text Add to dashboard Cite

show abstract

“…Visual feedback using 3D imaging allows surgeons to improve their endoscopic instrument control and can aid diagnosis of tissue abnormalities [15]. This can be achieved using a variety of techniques such as stereoscopic [16] and time of flight imaging [17] .…”

Section: Introductionmentioning

confidence: 99%

A combined fluorescence lifetime and depth imaging system for medical imaging and surgical guidance

Hopkinson

Matheson²,

Finlayson³

et al. 2023

Advanced Biomedical and Clinical Diagnostic and Surgical Guidance Systems XXI

View full text Add to dashboard Cite

Fluorescence lifetime imaging (FLIM) is a valuable technique which can be used to provide label free contrast between different tissue types and provide information about their molecular makeup and local environment. FLIM systems based on single photon avalanche diode (SPAD) arrays are increasingly being used in applications such as medical imaging due to their high sensitivity and excellent temporal resolution [1]. Additionally, SPAD arrays are also commonly employed for time of flight (ToF) imaging techniques such as light detection and ranging (LiDAR) [2]. Here we demonstrate a system which employs both of these modalities into a single instrument, allowing us to acquire both depth and widefield FLIM images simultaneously using a single 32 x 32 pixel SPAD array operating in time correlated single-photon counting (TCSPC) mode with 50 ps temporal resolution. Initial results show that we can correctly measure depths and distances of sample objects with < 1 cm resolution while maintaining excellent and consistent fluorescence contrast. Lifetime is consistent over a distance of 10 cm with a standard deviation of < 0.5 ns, showing that it is possible to decouple depth and lifetime data. We believe this work is the first demonstration of a widefield FLIM system capable of 3D imaging. The next step will be the addition of a miniaturized system [1] and future applications for this technology include fields such as surgical guidance, endoscopy and diagnostic imaging.

show abstract

Sub-millimeter precision 3D measurement through a standard endoscope with time of flight

Cited by 2 publications

References 20 publications

Dense 3D Reconstruction Through Lidar: A New Perspective on Computer-Integrated Surgery

Dense 3D Reconstruction Through Lidar: A New Perspective on Computer-Integrated Surgery

A combined fluorescence lifetime and depth imaging system for medical imaging and surgical guidance

Contact Info

Product

Resources

About