Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression

Abstract

PurposeMultispectral imaging can provide reflectance measurements at multiple spectral bands for each image pixel. These measurements can be used for estimation of important physiological parameters, such as oxygenation, which can provide indicators for the success of surgical treatment or the presence of abnormal tissue. The goal of this work was to develop a method to estimate physiological parameters in an accurate and rapid manner suited for modern high-resolution laparoscopic images.MethodsWhile previous methods for oxygenation estimation are based on either simple linear methods or complex model-based approaches exclusively suited for off-line processing, we propose a new approach that combines the high accuracy of model-based approaches with the speed and robustness of modern machine learning methods. Our concept is based on training random forest regressors using reflectance spectra generated with Monte Carlo simulations.ResultsAccording to extensive in silico and in vivo experiments, the method features higher accuracy and robustness than state-of-the-art online methods and is orders of magnitude faster than other nonlinear regression based methods.ConclusionOur current implementation allows for near real-time oxygenation estimation from megapixel multispectral images and is thus well suited for online tissue analysis.

Physiological Parameter Estimation from Multispectral Images Unleashed

“…Images were recorded with a custom-built, multispectral laparoscope, capturing images at eight different wavebands [3] with the 5Mpix Pixelteq (Largo, FL, USA) Spectrocam. We recorded MI data from seven pigs and six organs (liver, spleen, gallbladder, bowel, diaphragm and abdominal wall) in a laparoscopic setting.…”

“…A generalization of the layered tissue model developed in [3] was used to create our data set consisting of camera independent tissue-reflectance pairs (θ i , r i ), i ∈ {1...n}. The (physiological) parameters are varied within the ranges shown in Table 1 for each of the three layers.…”

“…The (physiological) parameters are varied within the ranges shown in Table 1 for each of the three layers. Each layer is described by its value for blood volume fraction v hb , scattering coefficient a mie , scattering power b mie , anisotropy g, refractive index n and layer thickness d. Oxygenation s is kept constant across layers [3].…”

“…Following the values in [5] and in contrast to [3], b mie is varied, covering all soft, fatty and fibrous tissues. We increase the ranges of v hb by a factor of three to potentially model pathologies.…”

“…Model-based approaches that are sufficiently fast for online execution typically suffer from incorrect base assumptions such as constant scattering or low tissue absorption [2]. Machine learning-based alternatives [3,4] need accurate information about the composition of the underlying tissue for the training phase to account for the lack of annotated real data. This knowledge about optical tissue properties is hard to obtain and is dependent on experimental conditions [5].…”