Joint Representation and Recognition for Ship-Radiated Noise Based on Multimodal Deep Learning

Abstract

Ship recognition based on ship-radiated noise is one of the most important and challenging subjects in underwater acoustic signal processing. The recognition methods for ship-radiated noise recognition include traditional methods and deep learning (DL) methods. Developing from the DL methods and inspired by audio–video speech recognition (AVSR), the paper further introduces multimodal deep learning (multimodal-DL) methods for the recognition of ship-radiated noise. In this paper, ship-radiated noise (acoustics modality) and visual observation of the ships (visual modality) are two different modalities that the multimodal-DL methods model on. The paper specially designs a multimodal-DL framework, the multimodal convolutional neural networks (multimodal-CNNs) for the recognition of ship-radiated noise. Then the paper proposes a strategy based on canonical correlation analysis (CCA-based strategy) to build a joint representation and recognition on the two different single-modality (acoustics modality and visual modality). The multimodal-CNNs and the CCA-based strategy are tested on real ship-radiated noise data recorded. Experimental results show that, using the CCA-based strategy, strong-discriminative information can be built from weak-discriminative information provided from a single-modality. Experimental results also show that as long as any one of the single-modalities can provide information for the recognition, the multimodal-DL methods can have a much better multiclass recognition performance than the DL methods. The paper also discusses the advantages and superiorities of the multimodal-Dl methods over the traditional methods for ship-radiated noise recognition.

Underwater Target Recognition Based on Multi-Decision LOFAR Spectrum Enhancement: A Deep-Learning Approach

“…After the emergence of the ShipsEar database, it has been used in the application research of ship radiated noise separation, denoising, classification, etc. It is also common to use this database to complete research in the field of deep learning [18][19][20][21][37][38][39].…”

Optimized Ship-Radiated Noise Feature Extraction Approaches Based on CEEMDAN and Slope Entropy

“…With the increasing complexity of the marine environment, the classification and identification of underwater acoustic targets are of great importance in areas such as national defense and the exploitation of marine resources [ 1 , 2 ]. A ship-radiated noise signal (S-NS), as the focus of research in the field of underwater acoustics, contains a variety of information such as ship target type, tonnage, speed, and so on, which is helpful in the recognition, classification, and tracking of ship targets [ 3 , 4 ]. The key technology of S-NS classification is “feature extraction”, and further development of the feature extraction technology is conducive to improving the classification performance of S-NSs [ 5 , 6 , 7 ].…”

Underwater acoustic target recognition method based on a joint neural network

“…Because the recognition framework of a single neural network makes the extraction of all features of underwater acoustic signals challenging [18][19][20], the research is usually focused on the development of deeper and more complex networks [21][22][23][24][25][26][27][28][29], which however are more difficult to train (in terms of training data size and labeling requirements). Therefore, building a new network model by combining various network structures may be a good solution.…”