A Framework for Multimedia Data Mining using Transformer based Intelligent DNN Model Architecture

Abstract

Multimedia data mining plays a crucial role in various fields, such as image and video analysis, natural language processing, and recommendation systems. Multimedia data refers to any form of data that involves multiple modes of communication, such as text, images, audio, and video. To effectively mine valuable insights from multimedia data, a new framework is proposed in this paper that employs a transformer-based intelligent deep neural network (DNN) model architecture. The framework includes an extensive data preprocessing step that involves obtaining multimedia data from internet searches and removing duplicates to ensure that each image is unique. The proposed transformer-based intelligent DNN model architecture processes the multimedia data in a hierarchical manner and utilizes shifted windows to achieve high accuracy in image classification task. The exploited dataset details are provided in the experimental evaluation section. Experimental results show that the proposed framework outperforms existing multimedia data mining methods in terms of accuracy and efficiency. This framework provides valuable insights that can be used in various applications, including content-based image retrieval, sentiment analysis, and automated captioning.