With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissions using the multimodal AI fusion techniques. Most of the gases and their fumes are colorless, odorless, and tasteless, thereby challenging our normal human senses. Sensing based on a single sensor may not be accurate, and sensor fusion is essential for robust and reliable detection in several real-world applications. We manually collected 6400 gas samples (1600 samples per class for four classes) using two specific sensors: the 7-semiconductor gas sensors array, and a thermal camera. The early fusion method of multimodal AI, is applied The network architecture consists of a feature extraction module for individual modality, which is then fused using a merged layer followed by a dense layer, which provides a single output for identifying the gas. We obtained the testing accuracy of 96% (for fused model) as opposed to individual model accuracies of 82% (based on Gas Sensor data using LSTM) and 93% (based on thermal images data using CNN model). Results demonstrate that the fusion of multiple sensors and modalities outperforms the outcome of a single sensor.

No abstract

The detection of gas leakages is a crucial aspect to be considered in the chemical industries, coal mines, home applications, etc. Early detection and identification of the type of gas is required to avoid damage to human lives and the environment. The MultimodalGasData presented in this paper is a novel collection of simultaneous data samples taken using seven different gas-detecting sensors and a thermal imaging camera. The low-cost sensors are generally less sensitive and less reliable; hence, they are unable to detect the gases from a longer distance. A thermal camera that can sense the temperature changes is also used while collecting the present multimodal dataset to overcome the drawback of using only the sensors for detecting gases. This multimodal dataset has a total of 6400 samples, including 1600 samples per class for smoke, perfume, a mixture of smoke and perfume, and a neutral environment. The dataset is helpful for the researchers and system developers to develop and train the state-of-the-art artificial intelligence models and systems.