The AI Mechanic: Acoustic Vehicle Characterization Neural Networks

Abstract

In a world increasingly dependent on road-based transportation, it is essential to understand vehicles. We introduce the AI mechanic, an acoustic vehicle characterization deep learning system, as an integrated approach using sound captured from mobile devices to enhance transparency and understanding of vehicles and their condition for non-expert users. We develop and implement novel cascading architectures for vehicle understanding, which we define as sequential, conditional, multilevel networks that process raw audio to extract highly-granular insights. To showcase the viability of cascading architectures, we build a multi-task convolutional neural network that predicts and cascades vehicle attributes to enhance fault detection. We train and test these models on a synthesized dataset reflecting more than 40 hours of augmented audio and achieve > 92% validation set accuracy on attributes (fuel type, engine configuration, cylinder count and aspiration type). Our cascading architecture additionally achieved 93.6% validation and 86.8% test set accuracy on misfire fault prediction, demonstrating margins of 16.4% / 7.8% and 4.2% / 1.5% improvement over naïve and parallel baselines. We explore experimental studies focused on acoustic features, data augmentation, feature fusion, and data reliability. Finally, we conclude with a discussion of broader implications, future directions, and application areas for this work.

Improving Misfire Fault Diagnosis with Cascading Architectures via Acoustic Vehicle Characterization

“…These experiments did not yield any notable trend and in turn have not included these results in our manuscript. Results exploring validation performance, confusion matrices, and hyperparameters can be found in [ 91 ]. However, any model weights and log files are available via request for reproduciblility.…”

Analysis of Unsupervised Learning Approach for Classification of Vehicle Fuel Type Using Psychoacoustic Features

“…This results in a stronger combustion process that produces a noticeable noise compared to combustion in gasoline engines. The importance and efficiency of artificial neural network-based classification for acoustic vehicle characterization has been demonstrated in many recent studies and literature reviews, [23][24][25][26]. Becker et al recognized the importance of vehicle engine sound classification as a privacy-friendly alternative to video-based vehicle classification.…”

“…They used a supervised neural network to classify the sounds and were able to achieve an average F1 score of 83 % when distinguishing between diesel and gasoline engines, [23]. The AI Mechanic presented the integration of a deep learning system for acoustic vehicle characterization using sound data collected from mobile devices, [24]. The tasks also included fuel type prediction.…”

“…In addition, the data set was augumented to produce over 2500 min of sound. Despite these efforts, predicting the fuel type still proved to be a difficult task, [24]. On the other hand, Wieczorkowska et al obtained better results with an F1 score of 96.2 % for diesel engines and 96 % for gasoline engines, [25].…”