Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Abstract

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. Spectral imaging takes advantage of the fact that different tissue components have unique optical properties to recover relevant information on tissue function such as ischemia. However, clinical success stories for advancing laparoscopic surgery with spectral imaging are lacking to date. To address this bottleneck, we developed the first laparoscopic real-time multispectral imaging (MSI) system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional RGB (Red, Green, and Blue) surgical view of the patient with functional information at a video rate of 25 Hz. To account for the high inter-patient variability of human tissue, we phrase the problem of ischemia detection as an out-of-distribution (OoD) detection problem that does not rely on data from any other patient. Using an ensemble of invertible neural networks (INNs) as a core component, our algorithm computes the likelihood of ischemia based on a short (several seconds) video sequence acquired at the beginning of each surgery. A first-in-human trial performed on 10 patients undergoing partial nephrectomy demonstrates the feasibility of our approach for fully-automatic live ischemia monitoring during laparoscopic surgery. Compared to the clinical state-of-the-art approach based on indocyanine green (ICG) fluorescence, the proposed MSI-based method does not require the injection of a contrast agent and is repeatable if the wrong segment has been clamped. Spectral imaging combined with advanced deep learning-based analysis tools could thus evolve as an important tool for fast, efficient, reliable and safe functional imaging in minimally invasive surgery.

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

“…Light source requirements such as available spectrum, long-term stability, switching behavior, or synchronization with the image sensor were fulfilled. Although comparable filter wheel 4 and snapshot 17 systems respectively reach higher spectral resolutions (FWHM: 10-25 nm versus 12 to 75 nm) and temporal resolutions (frame rate: 25 fps versus 20 fps), the presented MSI system benefits from a broader spectral range beyond the visible range and carries the potential for further applications like tissue discrimination.…”

“…Intraoperatively tissue perfusion is measured by clinical assessment lacking an objective gold standard. MSI has therefore been evaluated qualitatively 4 , 17 . In-vivo animal studies in which bowel tissue oxygenation is selectively reduced or in which tissue perfusion is imaged by fluorescence angiography should be performed in the future to investigate the added value and accuracy of MSI perfusion monitoring.…”

Development of a visible and over 1000 nm hyperspectral imaging rigid-scope system using supercontinuum light and an acousto-optic tunable filter

“…In addition, wavelength selection has the potential to realize real-time classification because the machine-learning calculation costs are reduced. If the classification is displayed at approximately 15-30 frames per second (video-like frame rate), the number of possible applications will increase, for example the on-sight measurement in industrial field and laparoscopic surgery can be expected [32].…”

A dual-camera hyperspectral laparoscopic imaging system

“…The recent development of high-speed HSI has begun to receive some attention [13], but generally the improved speed comes at the cost of reduced spectral and/or spatial dimension sizes. We seek to work towards addressing these drawbacks through the development of a dual-camera hyperspectral imaging setup for laparoscopic applications.…”