Spectral pre-processing for biomedical vibrational spectroscopy and microspectroscopic imaging

Abstract

IntroductionWith the development of modern analytical technologies such as infrared (IR) and Raman spectroscopy the capabilities of both generating and collecting data has been tremendously increased. Time-resolved vibrational spectroscopy, microspectroscopy, and vibrational hyperspectral imaging for example are now routinely employed in many areas of industry, technology development and scientific research. The advancements in IR and Raman instrumentation has led to an explosive growth in stored or transient data and has generated an urgent need for new and automated methods of spectral data analysis. Spectral data pre-processing is an important first step in the workflow of IR and Raman spectra analysis which involves specific processing procedures performed on the raw data. Pre-processing has been shown to be of crucial importance for subsequent data mining tasks. In fact, it is now widely recognized that quantitative and classification models developed on the basis of pre-processed data generally perform better than models that solely use raw data [1][2][3]. With this review it is intended to explore the concepts and techniques of pre-processing methods and to discuss the applicability of distinct pre-processing techniques in the field of biomedical IR and Raman spectroscopy. The main goals of data pre-processing can be summarized as follows: (i) Improvement of the robustness and accuracy of subsequent quantitative or classification analyses (ii) Improved interpretability: raw data are transformed into a format that will be better understandable by both humans and machines (iii) Detection and removal of outliers and trends (iv) Reduction of the dimensionality of the data mining task. Removal of irrelevant and redundant information by feature selection. For systematic reasons it is useful to subdivide data pre-processing procedures into at least eight different categories. These groups can be used to perform the following operations: 1. Exclusion (cleaning) -this class of data pre-processing methods is used to detect and eliminate spectral outliers. Tests for spectral quality are important examples and aim at the detection of outlier spectra prior to further data mining tasks. Removal of outlier spectra can be achieved by labeling them as NaNs (Not a Number), for example as "bad" pixel spectra in hyperspectral imaging applications. Another way of outlier removal in hyperspectral imaging applications is the interpolation of bad pixel spectra by using spectral data from neighboring locations. 2. Normalization -normalization is used to scale the spectra within a similar range. In vibrational spectroscopy this is useful to compensate for the differences in sample quantity or a different optical pathlength. In data mining tasks involving spectral distance measurements normalization has been shown to improve the accuracy and efficiency of the models [1][2][3]. 3. Filtering -in frequency analyses filtering is known as a group of methods that removes unwanted frequencies from the signals to be analyzed. Examples for...

Segmentation of Confocal Raman Microspectroscopic Imaging Data Using Edge-Preserving Denoising and Clustering

“…Instead of classical image analysis methods such as spatial filtering (e.g. sharpening, denoising), edge detection, segmentation and object recognition, HSI analysis methods currently rely predominantly on operations originally developed for point spectroscopy [20]. Vibrational hyperspectral imaging segmentation in biomedical applications is usually conducted via unsupervised spectral clustering [8,21], spectral unmixing [22][23][24], or supervised spectra classification [2,25].…”

Developing an Instrument-Independent Algorithm for Raman Spectroscopy: A Case of Cancer Detection

“…Employing the spectra for classification purposes requires some form of normalization that makes comparison between heterogeneous sets of samples more effectively (18,20). This is of high significance in our intended application in which different measurement systems were utilized.…”

“…In the normalization step, different normalization methods (18) [including min-max, 1-norm, 2-norm and SNV (21)] were evaluated at different pre-processing stages i.e., on raw spectra, after resolution matching or after noise and background elimination.…”

Review of multidimensional data processing approaches for Raman and infrared spectroscopy

“…Such techniques include Min-max normalization, SNV normalization etc. These methods are applied individually to each spectrum, so these methods can be classified as 1-way methods [46]. Certain normalization methods considering more than one spectrum (say 'n' number of spectra) at a time while building the model can be classified as n-way methods.…”

“…a) Both upper and lower thresholds can be applied on FTIR absorbance values of a specific vibration mode, for example the amide I region, which generally indicates inconsistent sample thickness regions [46,51]. For example, a low sample thickness is indicated by the presence of noise in the case of FTIR imaging data.…”