Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

Abstract

With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissions using the multimodal AI fusion techniques. Most of the gases and their fumes are colorless, odorless, and tasteless, thereby challenging our normal human senses. Sensing based on a single sensor may not be accurate, and sensor fusion is essential for robust and reliable detection in several real-world applications. We manually collected 6400 gas samples (1600 samples per class for four classes) using two specific sensors: the 7-semiconductor gas sensors array, and a thermal camera. The early fusion method of multimodal AI, is applied The network architecture consists of a feature extraction module for individual modality, which is then fused using a merged layer followed by a dense layer, which provides a single output for identifying the gas. We obtained the testing accuracy of 96% (for fused model) as opposed to individual model accuracies of 82% (based on Gas Sensor data using LSTM) and 93% (based on thermal images data using CNN model). Results demonstrate that the fusion of multiple sensors and modalities outperforms the outcome of a single sensor.

[Retracted] Vision Sensor‐Based Real‐Time Fire Detection in Resource‐Constrained IoT Environments

“…CNNs are widely used and in complex visual recognition tasks such as action and activity recognition [ 41 ], anomaly detection and recognition [ 42 , 43 ], classification [ 44 , 45 ], object detection [ 46 ], and a variety of other recognition, video summarization, and segmentation tasks [ 41 – 49 ]. The CNN architecture consists of convolutional layers (CL), pooling layers, and fully connected layers.…”

Employing Multimodal Machine Learning for Stress Detection

“…In [ 5 ], Gros Dut presents an in-depth analysis of the logic underpinning data fusion and discusses data fusion and multisensor integration approaches. Narkhede et al [ 6 ] propose a method to detect gaseous emissions using multimodal data collected from gas sensors and thermal cameras. The fused model achieved 96% accuracy on the testing set instead of 82% on LSTM applied to sensor data and 93% on CNN applied to camera images for individual modalities.…”

Real-Time Human Recognition at Night via Integrated Face and Gait Recognition Technologies

“…Surpassing human-level performance propelled the research in applications where different modalities amongst language, vision, sensory, text play an essential role inaccurate predictions and identification [ 45 ]. Several state-of-the-art approaches in multimodal fusion employing deep learning models are proposed in the literature, such as approaches presented by F. Ramzan et al [ 24 ], A. Zlatintsi et al [ 25 ], M. Dhouib and S. Masmoudi [ 26 ], Y. D. Zhang et al [ 27 ], C. Devaguptapu et al [ 28 ], and P. Narkhede et al [ 46 ]. The purpose of these approaches is to enhance the multimodal fusion method with which the objects can be efficiently detected from static images or given video sequences with the preferable use of the deep learning library.…”