BERT-hLSTMs: BERT and hierarchical LSTMs for visual storytelling

Su, Jing; Dai, Qingyun; Guérin, Frank; Zhou, Mian

doi:10.1016/j.csl.2020.101169

Cited by 24 publications

(6 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in LSTM architecture, there are 4 layers in the loop as shown in Fig. 1 [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29]. Bidirectional Long Short-Term Memory (Bi-LSTM) is a variant of Recurrent Neural Network (RNN).…”

Section: Long Short-term Memory (Lstm)mentioning

confidence: 99%

Social Computing to Create Government Public Policy Document Blueprint Draft Based on Social Media Data About Covid-19 Using LSTM and MMR Hybrid Algorithms

Cholissodin

2021

ICGT

View full text Add to dashboard Cite

Abstract- Determining a policy is often limited in a short time so that decisions are prone to inaccuracies and are ultimately judged to be less targeted. Therefore, there is a necessity to use data mining technology. Currently, especially due to the continuously increasing case of the spread of COVID-19 in Indonesia, in order to reduce the rate of spread of COVID-19, the government has established a COVID-19 vaccination and emergency Community Activity Restriction Implementation (PPKM) policies. For the success of the policies, the government is required to ascertain and understand the attitude of the society. Hence, the policies can be accepted and supported by the society. YouTube is one of the sources to discover people’s attitudes because in YouTube, people can express their opinions freely. In this study, a model based on Natural Language Processing (NLP) with the Deep Learning method was developed to analyze people’s attitudes from their writings or posts on social media. As for the algorithm stages, first the model analysis was created using the Long Short-Term Memory (LSTM) and Bidirectional Long Short-Term Memory (Bi-LSTM) algorithms as a comparison. In the COVID-19 vaccination policy, the Bi-LSTM algorithm provided a better evaluation value, i.e., the accuracy value of 87.20%, recall of 87.14%, precision of 87.14%, and F1-score of 87.14%. In the emergency PPKM policy, the LSTM algorithm provided a better evaluation value of 95.13%, recall of 93.94%, precision of 94.08%, and F1-score of 94.01%. In addition, using the Maximum Marginal Relevance (MMR) method to obtain recommendations related to the COVID-19 vaccination policy, the result showed that the government needs to carry out re-socialization regarding vaccination and the impact of the vaccine so that the society can become more cooperative and the emergency PPKM policy needs to be reviewed because it has an impact on the society’s economy.

show abstract

Section: Long Short-term Memory (Lstm)mentioning

confidence: 99%

Social Computing to Create Government Public Policy Document Blueprint Draft Based on Social Media Data About Covid-19 Using LSTM and MMR Hybrid Algorithms

Cholissodin

2021

ICGT

View full text Add to dashboard Cite

show abstract

“…This model is implemented based on bidirectional transformer encoder [20] , breaking 11 task records of natural language processing, and is now widely used in text classification, natural language understanding, machine translation, etc. [21][22][23] Literature [24] proposed the electric BERT pre-training model by training the electric power corpus. Through it, the transformer defect text was converted into word vector, and the entity recognition model was established based on BiLSTM-CRF, which greatly improved the accuracy of entity recognition of transformer defect text.…”

Section: Introductionmentioning

confidence: 99%

The entity and event recognition method of power dispatching text information based on BERT-CRF

Liang

Dai

You

et al. 2023

Third International Conference on Artificial Intelligence and Computer Engineering (ICAICE 2022)

View full text Add to dashboard Cite

In order to improve the accuracy of power dispatching text analysis and the ability to guide the operation of the power grid, a power dispatch text entity recognition method is proposed based on bidirectional encoder representations from transformers-conditional random field (BERT-CRF). Taking the power grid fault handling plan text as the research object, the entity marking method of the fault handling plan is propose. The word vector of the plan entity is calculated based on the BERT pre-training model, the characterization ability of the professional entity of the plan is enhanced by fine-tuning the initial BERT parameters, and the recognition ability of the plan text sequence is improved from the overall situation to access the CRF layer in the neural network. Thus, an entity recognition model of fault handling plan is established based on the BERT-CRF. Through the verification of a power grid fault handling plan, the proposed method has higher power dispatch entity and event recognition accuracy compared with other algorithms.

show abstract

“…Early work [1] proposed a hierarchically‐attentive recurrent neural network, which included three RNNs, to generate stories. The researchers in [2] developed an end‐to‐end Bert‐hLSTMs model comprised of Bert and hierarchical LSTMs, in order to capture both sentence‐level and word‐level semantic information. Based on graph knowledge and relational reasoning, IRW [3] mimicked the logic of human story writing to create stories.…”

Section: Introductionmentioning

confidence: 99%

Video storytelling based on gated video memorability filtering

2022

Electronics Letters

View full text Add to dashboard Cite

Video storytelling aims to generate a successive and succinct paragraph composed of multiple sentences to describe the visual contents in a long video. The major difficulty with video storytelling is the conflict between complex visual semantics and concise textual description. Existing methods address this problem by applying different segmentation strategies to long videos. However, these methods ignore that some story-irrelevant visual contents could be considered as noises, so visual information in a long video is not always conducive to story generation. Therefore, we propose a novel video storytelling model based on gated video memorability filtering (GVMF), which generates highquality textual descriptions by filtering out clips with lower memorability scores, where objects and characters appear less frequently. The reason for introducing video memorability prediction is that video clips with higher video memorability scores tend to have more characters and objects so tend to be noticed by annotators for writing sentences. Experimental results demonstrate the effectiveness and competitiveness of our method, which achieves the new state-of-the-art results on Video Story dataset.

show abstract

BERT-hLSTMs: BERT and hierarchical LSTMs for visual storytelling

Cited by 24 publications

References 45 publications

Social Computing to Create Government Public Policy Document Blueprint Draft Based on Social Media Data About Covid-19 Using LSTM and MMR Hybrid Algorithms

Social Computing to Create Government Public Policy Document Blueprint Draft Based on Social Media Data About Covid-19 Using LSTM and MMR Hybrid Algorithms

The entity and event recognition method of power dispatching text information based on BERT-CRF

Video storytelling based on gated video memorability filtering

Contact Info

Product

Resources

About