Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks

Abstract

In recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasive substance classification. One of the areas of potential application is forensic science, where substance classification on the scenes is important. An example problem from that area—blood stain classification—is a case study for the evaluation of methods that process hyperspectral data. To investigate the deep learning classification performance for this problem we have performed experiments on a dataset which has not been previously tested using this kind of model. This dataset consists of several images with blood and blood-like substances like ketchup, tomato concentrate, artificial blood, etc. To test both the classic approach to hyperspectral classification and a more realistic application-oriented scenario, we have prepared two different sets of experiments. In the first one, Hyperspectral Transductive Classification (HTC), both a training and a test set come from the same image. In the second one, Hyperspectral Inductive Classification (HIC), a test set is derived from a different image, which is more challenging for classifiers but more useful from the point of view of forensic investigators. We conducted the study using several architectures like 1D, 2D and 3D convolutional neural networks (CNN), a recurrent neural network (RNN) and a multilayer perceptron (MLP). The performance of the models was compared with baseline results of Support Vector Machine (SVM). We have also presented a model evaluation method based on t-SNE and confusion matrix analysis that allows us to detect and eliminate some cases of model undertraining. Our results show that in the transductive case, all models, including the MLP and the SVM, have comparative performance, with no clear advantage of deep learning models. The Overall Accuracy range across all models is 98–100% for the easier image set, and 74–94% for the more difficult one. However, in a more challenging inductive case, selected deep learning architectures offer a significant advantage; their best Overall Accuracy is in the range of 57–71%, improving the baseline set by the non-deep models by up to 9 percentage points. We have presented a detailed analysis of results and a discussion, including a summary of conclusions for each tested architecture. An analysis of per-class errors shows that the score for each class is highly model-dependent. Considering this and the fact that the best performing models come from two different architecture families (3D CNN and RNN), our results suggest that tailoring the deep neural network architecture to hyperspectral data is still an open problem.

Trends in Deep Learning for Medical Hyperspectral Image Analysis

“…Cell classification using convolutional neural networks in medical hyperspectral imagery [45] Classification Hyperspectral imaging for cancer detection and classification [47] Medical hyperspectral imaging: a review [7] Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging [35] Deep learning based classification for head and neck cancer detection with hyperspectral imaging in an animal model [55] Convolutional neural network for medical hyperspectral image classification with kernel fusion [51] Classification Blood cell classification based on hyperspectral imaging with modulated Gabor and CNN [52] Medical hyperspectral image classification based on end-to-end fusion deep neural network [53] Tissue classification of oncologic esophageal resectates based on hyperspectral data [49] Computer-assisted medical image classification for early diagnosis of oral cancer employing deep learning algorithm [57] Design of a multilayer neural network for the classification of skin ulcers' hyperspectral images: a proof of concept [48] Hyperspectral imaging based method for fast characterization of kidney stone types [46] Design of a multilayer neural network for the classification of skin ulcers' hyperspectral images: a proof of concept [83] Non-invasive skin cancer diagnosis using hyperspectral imaging for in-situ clinical support [50] Hyperspectral imaging for colon cancer classification in surgical specimens: towards optical biopsy during image-guided surgery [84] Deep learning applied to hyperspectral endoscopy for online spectral classification [85] Spectral-spatial recurrent-convolutional networks for in-vivo hyperspectral tumor type classification [56] Blood stain classification with hyperspectral imaging and deep neural networks [54] Hyperspectral imaging for glioblastoma surgery: improving tumor identification using a deep spectral-spatial approach [58] Dual-modality endoscopic probe for tissue surface shape reconstruction and hyperspectral imaging enabled by deep neural networks [62] Detection Probe-based rapid hybrid hyperspectral and tissue surface imaging aided by fully convolutional networks [63] A dual stream network for tumor detection in hyperspectral images [59] Adaptive deep learning for head and neck cancer detection using hyperspectral imaging [67] Detection Hyperspectral imaging of head and neck squamous cell carcinoma for can...…”

“…Hyperspectral system for imaging of skin chromophores and blood oxygenation [79] Other Estimation of tissue oxygen saturation from RGB images and sparse hyperspectral signals based on conditional generative adversarial network [78] Conditional generative adversarial network for synthesizing hyperspectral images of breast cancer cells from digitized histology [81] Generating hyperspectral skin cancer imagery using generative adversarial neural network [82] CNN-based model used for endoscopic image reconstruction to enhance surgical guidance [62] Classifying cancerous tissue samples from neck and head regions using CNN [35] Detection of neck and head cancerous cells via classification using CNN [55] Improvisation of CNN using kernel fusion implemented for cell classification [51] Implementation of CNN for blood cell classification [52] Two-channel CNN for solving limited-samples problem for CNN models [53] Use of CNN to detect squamous cell carcinoma between samples from different patients [65] CNN used for detection of oral cancer [57] Using specular glare in MHSI along with CNN to detect squamous cell carcinoma [66] Another study for CNN to detect squamous cell carcinoma [61] Detection of brain tumor with the aid of CNN [71] CNN Detecting carcinoma thyroid sample with the aid of CNN [60] Different CNN models compared to one another for classifying skin cancer from patient data HIS [72] CNN utilized to classify and detect squamous cell carcinoma [68] CNN used to classify and detect breast cancer cells [70] CNN implemented for detection of Glioblastoma cells from Hematoxylin & Eosin tissue sample [86] In-vivo Laryngeal cancer detection based on CNN [69] Convolutional based RetinaNet model implemented to classify and detect tumors in epithelial tissue [87] Implemented a hybrid CNN model to classify colon tumor in order to aid surgical guidance [84] Five-layer CNN applied to classify endoscopy HSI [85] Study of classification for blood and similar appearing substances from HSI with CNN,RNN, MLP [54] Proposed framework of 3D-2D CNN-based approach to classify brain tumors [58] ANN Implementation of ANN and SVM for cancerous cell HSI <...>…”

Adaptive Detection of Direct-Sequence Spread-Spectrum Signals Based on Knowledge-Enhanced Compressive Measurements and Artificial Neural Networks

“…To train the neural network to get the measurement kernel of the mth measurement in the multiple neural network strategy, the training penalty function is taken to be the negativity of the conditional differential entropy in Equation ( 15), i.e., −h(y m |H 1 , ℵ m−1 , A m ). Such a penalty function depends on the posterior probabilities of the PN sequence usage and the designed measurement kernel, i.e., P b (l|H 1 , ℵ m−1 ) and A m , according to Equation (15). In the single neural network strategy, the training penalty function is then taken to be −…”

“…Presently, with the rapid development of the computer technologies, especially the parallel computing, the artificial neural network (or neural network) and the deep learning [ 14 ] techniques have been widely used in the area of signal processing, such as topics on biomedical and civil engineering [ 15 , 16 ]. The well-trained neural networks can efficiently extract the features of the signals, and are proved to have good performance in pattern recognition, signal parameter estimation, prediction, etc.…”

Hyperspectral Imaging for Bloodstain Identification

“…Different traces that are found in crime casework can be very important. A schematic search, analysis, and conclusions from these traces make them valuable in court investigations [ 1 , 2 , 3 , 4 ]. One of the most important forms of forensic shreds of evidence found at a crime scene is body fluids.…”