Robust Audio-Image Steganography using Cross-Modal Based Transformer Models

Abstract

This research investigates the use of Vision Transformers (ViT), Audio Spectrogram Transformers (AST), and Cross-Modal Transformers (CMT) in audio-image fusion tasks, aiming to improve the representation learning and interaction between auditory and visual data. The ViT model extracts visual features from image patches resized to 224x224 pixels, while the AST model converts audio signals into mel spectrograms to capture detailed auditory features. The central focus is on the robust CMT model, which integrates visual and auditory features through a cross-modal attention mechanism. Extensive experiments using a diverse audio-image dataset from Kaggle reveal significant improvements. The initial developed ViT model enhances image embedding capacity by 12%, the AST model improves audio embedding tasks by 15%, and the CMT model achieves and overall embedding capacity increased by an average of 58.7%, an image PSNR improved by an average of 57.5%, image MSE reduced by an average of 87.3% as the CMT model shows more consistent performance across all the different stegos. this enhancement in cross-modal retrieval tasks, highlighting the effectiveness of robust CMT model in learning and utilizing inter-modal relationships. The proposed robust cross-modal attention mechanism outperforms traditional ViT model and Least Significant Bits [LSB] concatenation algorithms in feature alignment accuracy. Ablation studies further validate the robustness of this approach, demonstrating the contribution of each component to overall performance. This research establishes the viability and superiority of transformer-based architectures in audio-image fusion tasks, suggesting potential advancements in cross-modal tasks like audio-visual synchronization and multimodal sentiment analysis.