GIT: A Transformer-Based Deep Learning Model for Geoacoustic Inversion

Abstract

Geoacoustic inversion is a challenging task in marine research due to the complex environment and acoustic propagation mechanisms. With the rapid development of deep learning, various designs of neural networks have been proposed to solve this issue with satisfactory results. As a data-driven method, deep learning networks aim to approximate the inverse function of acoustic propagation by extracting knowledge from multiple replicas, outperforming conventional inversion methods. However, existing deep learning networks, mainly incorporating stacked convolution and fully connected neural networks, are simple and may neglect some meaningful information. To extend the network backbone for geoacoustic inversion, this paper proposes a transformer-based geoacoustic inversion model with additional frequency and sensor 2-D positional embedding to perceive more information from the acoustic input. The simulation experimental results indicate that our proposed model achieves comparable inversion results with the existing inversion networks, demonstrating its effectiveness in marine research.

Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review

“…Table 2 illustrates the distribution of published articles focusing on the application of the transformer architecture and the attention mechanism for genome data, across a variety of scientific journals. (20), constituting 16.1% of the total studies in this domain. The 'Bioinformatics', 'BMC Bioinformatics', and 'Frontiers in Genetics' journals follow closely, each contributing 7.3% of the total publications.…”

“…Inspired by the success of attention mechanisms, the transformer model was proposed as a complete shift from the sequential processing nature of recurrent neural networks (RNNs) and their variants [19][20][21][22]. The transformer model leverages attention mechanisms to process the input data in parallel, allowing for faster and more efficient computations.…”