Semi-Supervised Machine Condition Monitoring by Learning Deep Discriminative Audio Features

Abstract

In this study, we aim to learn highly descriptive representations for a wide set of machinery sounds and exploit this knowledge to perform condition monitoring of mechanical equipment. We propose a comprehensive feature learning approach that operates on raw audio, by supervising the formation of salient audio embeddings in latent states of a deep temporal convolutional neural network. By fusing the supervised feature learning approach with an unsupervised deep one-class neural network, we are able to model the characteristics of each source and implicitly detect anomalies in different operational states of industrial machines. Moreover, we enable the exploitation of spatial audio information in the learning process, by formulating a novel front-end processing strategy for circular microphone arrays. Experimental results on the MIMII dataset demonstrate the effectiveness of the proposed method, reaching a state-of-the-art mean AUC score of 91.0%. Anomaly detection performance is significantly improved by incorporating multi-channel audio data in the feature extraction process, as well as training the convolutional neural network on the spatially invariant front-end. Finally, the proposed semi-supervised approach allows the concise modeling of normal machine conditions and accurately detects system anomalies, compared to existing anomaly detection methods.

An EfficientNet-Based Weighted Ensemble Model for Industrial Machine Malfunction Detection Using Acoustic Signals

“…This illustrates that the proposed model significantly outperforms all existing models for both tested machines (i.e., pumps and valves). The malfunction detection in terms of the performance of an AUC increases by up to 9.51% and 3.23% for the pumps and valves, respectively, compared to the newest model developed this year by Thoidis et al [41]. This improvement is also supported by the F1 value of pumps compared to a previous model that increased malfunction detection by 9.56%.…”

“…In this experiment, it is shown that the proposed model is effective with both pump and valve machine types due to balanced AUC between pumps and valves (less than 1%), while other existing models, such as SPIDERnet, AE by Purohit et al [19], and MDF-FCN, are more suited for pumps than valves. Talmoudi and Hirata [42], Ribeiro et al [6] and RawdNet by Thoidis et al [41] are more suited for valves. These are shown due to their performance of an AUC that differs from 3.7% up to 21.5% for pumps compared to valves.…”

“…Classification results (e.g., SVM) demonstrate a superior AUC score when compared to classical machine learning models applied to the same dataset while maintaining a small model parameter set for acceptable generalization. Finally, Thoidis et al [41] provide a method for extracting discriminative embeddings from multi-channel raw audio for diverse machinery sounds. To identify problems on specific machines, a deep CNN learns machine embeddings and transfers them to a deep one-class neural network.…”

Robust and Lightweight Deep Learning Model for Industrial Fault Diagnosis in Low-Quality and Noisy Data

“…Studies using semi-supervised, unsupervised, and self-supervised learning methods include outlier detection, using self-supervised complex networks [12], outlier detection, using an unsupervised domain adaptation method [13], a study on a learning MIMII dataset through the semi-supervised learning of the RawdNet model [14], and an outlier detection study using self-supervised learning that utilizes a contrast learning framework [15]. MIMII dataset preprocessing has been realized by removing noise from MIMII datasets, via NMF and nnCP models [16].…”