Opti-MSFA: a toolbox for generalized design and optimization of multispectral filter arrays

Abstract

Multispectral imaging captures spatial information across a set of discrete spectral channels and is widely utilized across diverse applications such as remote sensing, industrial inspection, and biomedical imaging. Multispectral filter arrays (MSFAs) are filter mosaics integrated atop image sensors that facilitate cost-effective, compact, snapshot multispectral imaging. MSFAs are pre-configured based on application—where filter channels are selected corresponding to targeted absorption spectra—making the design of optimal MSFAs vital for a given application. Despite the availability of many design and optimization approaches for spectral channel selection and spatial arrangement, major limitations remain. There are few robust approaches for joint spectral-spatial optimization, techniques are typically only applicable to limited datasets and most critically, are not available for general use and improvement by the wider community. Here, we reconcile current MSFA design techniques and present Opti-MSFA: a Python-based open-access toolbox for the centralized design and optimization of MSFAs. Opti-MSFA incorporates established spectral-spatial optimization algorithms, such as gradient descent and simulated annealing, multispectral-RGB image reconstruction, and is applicable to user-defined input of spatial-spectral datasets or imagery. We demonstrate the utility of the toolbox by comparing against other published MSFAs using the standard hyperspectral datasets Samson and Jasper Ridge, and further show application on experimentally acquired fluorescence imaging data. In conjunction with end-user input and collaboration, we foresee the continued development of Opti-MSFA for the benefit of the wider research community.

Noninvasive hemoglobin sensing and imaging: optical tools for disease diagnosis

“… 171 , 181 , 202 , 203 Sometimes, a combination of classification and unmixing techniques can produce the optimal results, allowing for data corrections to be applied in certain tissue types. 26 , 76 , 166 , 170 Similar methods are also used in depth-resolved imaging, but data may need to be corrected based on the imaging depth and the associated level of optical absorption and scattering. Machine learning methods have shown promise in this regard, enabling more accurate determination of hemoglobin oxygenation, particularly at depth, than classic linear spectral unmixing.…”

“…The result is a 3D dataset ( ) 26 , 72 , 159 , 163 that can be subjected to multivariate analysis methods to extract from the measured spectra the concentrations of their contributing chromophores (e.g., Hb and ), referred to as “endmembers” for unmixing. 26 , 72 , 164 – 167 From these multivariate analyses, biomarkers that relate to THb and can then be extracted.…”

“…Nonetheless, optimization of the target wavelengths for snapshot imagers can substantially improve their performance. 166 , 184 – 187 …”

“…Nonetheless, optimization of the target wavelengths for snapshot imagers can substantially improve their performance. 166,[184][185][186][187] In addition to these intensity-based imaging methods, spatial frequency domain imaging (SFDI) can be used to interrogate hemoglobin-related biomarkers by exploiting modulated illumination of the tissue. SFDI typically uses sinusoidal patterns and extracts the demodulation of these patterns reflected from tissue to calculate absorption and scattering properties; 123,188 low frequencies are more sensitive to absorption whereas high frequencies are more sensitive to scattering.…”

Band selection for mapping chromophores of skin tissue

“…They found three or seven bands optimally combined were sufficient for estimating tissue hemoglobin oxygenation [11]. Sawyer et al found the performance of estimating oxygen saturation was particularly sensitive to those specific wavelengths such as isobestic bands, HbO 2 and Hb peaks, respectively [12]. Marois et al selected bands by maximizing the product of the singular values of absorption matrix associating with the absorbers like water, oxygenated and deoxygenated hemoglobin.…”