VNIR–NIR hyperspectral imaging fusion targeting intraoperative brain cancer detection

Abstract

Currently, intraoperative guidance tools used for brain tumor resection assistance during surgery have several limitations. Hyperspectral (HS) imaging is arising as a novel imaging technique that could offer new capabilities to delineate brain tumor tissue in surgical-time. However, the HS acquisition systems have some limitations regarding spatial and spectral resolution depending on the spectral range to be captured. Image fusion techniques combine information from different sensors to obtain an HS cube with improved spatial and spectral resolution. This paper describes the contributions to HS image fusion using two push-broom HS cameras, covering the visual and near-infrared (VNIR) [400–1000 nm] and near-infrared (NIR) [900–1700 nm] spectral ranges, which are integrated into an intraoperative HS acquisition system developed to delineate brain tumor tissue during neurosurgical procedures. Both HS images were registered using intensity-based and feature-based techniques with different geometric transformations to perform the HS image fusion, obtaining an HS cube with wide spectral range [435–1638 nm]. Four HS datasets were captured to verify the image registration and the fusion process. Moreover, segmentation and classification methods were evaluated to compare the performance results between the use of the VNIR and NIR data, independently, with respect to the fused data. The results reveal that the proposed methodology for fusing VNIR–NIR data improves the classification results up to 21% of accuracy with respect to the use of each data modality independently, depending on the targeted classification problem.

Intraoperative tissue classification methods in orthopedic and neurological surgeries: A systematic review

“…Six publications ( 9 , 14 – 18 ) were found in this category that focused on tissue classification based on hyperspectral imaging ( Table 2 ). These articles focused on detecting brain tumor tissue since hyperspectral imaging facilitates non-contact, non-ionizing, and real-time data acquisition that is used for tumor diagnosis.…”

“…Extracted reports used different technologies of varying spectral wavelengths. The majority of studies used Visual and Near-Infrared (VNIR) push-broom cameras covering a 400–1,000 nm wavelength range ( 9 , 14 , 15 , 17 , 18 ). In contrast to push-broom imaging technology, Urbanos et al ( 16 ) used a VNIR snapshot camera that can be used in a surgical environment due to its smaller footprint.…”

“…However, such sensors generally deliver a narrower wavelength range, namely 655–955 nm. Leon et al ( 14 ) proposed the combination of VNIR and Near-Infrared (NIR) cameras to cover the broadband spectral range between 435 nm and 1638 nm.…”

Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery

“…The continuous improvement of HSI systems in imaging, spatial resolution, spectral range, and resolution will enable portable and smaller HSI devices, and facilitate widespread medical applications in cerebral diagnosis. In addition, scholars have also begun to use multiple HS cameras to acquire more images in the broad spectral band, with the aim of analyzing different characteristics of different types of normal brain and tumor tissue in vivo ( Leon et al, 2021a , Leon et al, 2021b ). 3) Real-time processing of HSI video.…”

Evaluation of Hyperspectral Imaging for Follow-Up Assessment after Revascularization in Peripheral Artery Disease

“…In visceral surgery, it is used to assess anastomosis in the upper [25] and lower gastrointestinal tract [26][27][28]. In neurosurgery, HSI can be used for intraoperative brain cancer targeting [29][30][31]. In reconstructive surgery, HSI is feasible for identifying cutaneous perforators for microvascular surgery [32] and for perioperative perfusion monitoring of free and pedicled flaps [33,34].…”