Att-BiL-SL: Attention-Based Bi-LSTM and Sequential LSTM for Describing Video in the Textual Formation

Abstract

With the advancement of the technological field, day by day, people from around the world are having easier access to internet abled devices, and as a result, video data is growing rapidly. The increase of portable devices such as various action cameras, mobile cameras, motion cameras, etc., can also be considered for the faster growth of video data. Data from these multiple sources need more maintenance to process for various usages according to the needs. By considering these enormous amounts of video data, it cannot be navigated fully by the end-users. Throughout recent times, many research works have been done to generate descriptions from the images or visual scene recordings to address the mentioned issue. This description generation, also known as video captioning, is more complex than single image captioning. Various advanced neural networks have been used in various studies to perform video captioning. In this paper, we propose an attention-based Bi-LSTM and sequential LSTM (Att-BiL-SL) encoder-decoder model for describing the video in textual format. The model consists of two-layer attention-based bi-LSTM and one-layer sequential LSTM for video captioning. The model also extracts the universal and native temporal features from the video frames for smooth sentence generation from optical frames. This paper includes the word embedding with a soft attention mechanism and a beam search optimization algorithm to generate qualitative results. It is found that the architecture proposed in this paper performs better than various existing state of the art models.

Medical equipment effectiveness evaluation model based on cone-constrained DEA and attention-based bi-LSTM

“…Building on Bi-LSTM networks, integrating attention mechanisms 40 can further enhance the identification and modelling of critical features in time series (Fig. 2 ), thereby improving the temporal prediction task performance.…”

Implementasi Bidirectional LSTM untuk Analisis Sentimen Terhadap Layanan Grab Indonesia

“…LSTM memiliki kelemahan hanya memproses kata satu satu arah saja, yaitu dari awal sampai akhir [18]. Kami dalam penelitian ini mempertimbangkan untuk menggunakan bidirectional LSTM agar dapat membaca kata dari 2 arah, yaitu awal sampai akhir dan akhir sampai awal [19]. Sesuai arstitektur bidirectional LSTM pada gambar 4, bidirectional LSTM memilki 2 lapisan LSTM yang terpisah satu untuk maju dan satu untuk mundur sesuai dengan persamaan 7.…”

Bi-LS-AttM: A Bidirectional LSTM and Attention Mechanism Model for Improving Image Captioning

“…The disadvantage of these methods is the low efficiency of generating real and accurate sentences and the poor ability to generate structurally novel sentences. In recent research [7][8][9][10][11], visual and language information has been embedded into a common space via recurrent neural networks (RNNs) initially. Convolutional neural networks (CNNs) were then embedded within the visual space and combined with long short-term memory (LSTM) to produce more effective results.…”