Multiple premises entailment recognition based on attention and gate mechanism

Wu, Pin; Lei, Zhidan; Zhou, Quan; Zhu, Rukang; Chang, Xuting; Sun, Junwu; Zhang, Wenjie; Guo, Yike

doi:10.1016/j.eswa.2020.113214

Cited by 6 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the same time, in the process of model training, facing the pain points such as limited data volume of cold chain logistics prediction, most of the existing researches use Attention mechanism to train the prediction model. Although Attention mechanism is widely used in AI and other scenarios as the mainstream, it usually needs a large amount of data to learn reasonable weight parameters, and cannot learn the sequence relationship in the sequence [17]. Therefore, the Attention mechanism may not be effective in predicting continuous values with limited data.…”

Section: Innovationmentioning

confidence: 99%

Cold Chain Logistics Demand Forecasting Based on Improved BP Neural Network Model

Deng,

Zhang,

Wang

2024

Advances in Transdisciplinary Engineering

View full text Add to dashboard Cite

According to the characteristics of continuous value prediction and the imitation of the sample data size of cold chain logistics demand, an improved BP neural network model based on gating mechanism and GELU is proposed. Through gating mechanism, neural network pays more attention to important features in input, and GELU is used as the activation function of neural network, which can provide better nonlinear ability. In the empirical process, the model is used to forecast the demand of cold chain logistics of agricultural products in Hubei Province. The improved BP neural network achieves 96.07% similarity on the test dataset, which represents the state of the art in cold chain logistics demand forecast.

show abstract

Section: Innovationmentioning

confidence: 99%

Cold Chain Logistics Demand Forecasting Based on Improved BP Neural Network Model

Deng,

Zhang,

Wang

2024

Advances in Transdisciplinary Engineering

View full text Add to dashboard Cite

show abstract

“…In addition, the authors in [21] propose a model for the task of multiplepremise inference that uses a gate mechanism and use multiple LSTMs. Another model for multiplepremise inference is proposed in [30] by applying local matching-integration and concatenate-matching. In local matching, each premise sentence is individually matched with the hypothesis sentence to obtain multiple local classi cation results, which are then merged to get the nal inference classi cation.…”

Section: Related Workmentioning

confidence: 99%

Investigating the Use of Different Recurrent Neural Networks for Natural Language Inference in Arabic

Ben-Sghaier¹,

Bakari²,

Néji³

2023

Preprint

View full text Add to dashboard Cite

Natural language inference (NLI) is a subfield of natural language processing (NLP) that involves determining the logical relationship between two pieces of text, usually a premise and a hypothesis. The goal of an NLI system is to classify the inference relationship between the premise and the hypothesis into one of three categories, namely, entailment, contradiction and neutral. An understanding of this relationship is useful for performing several NLP tasks, such as, summarization, question-answering, information retrieval, etc. Given the potential of neural networks to handle complex natural language tasks, and the absence of prior research on their application to NLI in Arabic. In this context, we propose the investigation of different neural network models for the NLI task in Arabic language. Particularly, we carried out experiments using various types of recurrent neural networks, such as, Simple RNN, LSTM, GRU, Bi-LSTM, and BiGRU to find the one that performs best on the NLI task in Arabic. We performed our experiments using the existing datasets for Arabic NLI namely, ArbTE, XNLI, and ArNLI.

show abstract

“…In practical applications, RNN has been able to process some simple correlation information while its memory capacity is not strong. When the sequence is too long, error back propagation will cause larger gradient dispersion and gradient explosion problems, which can be effectively alleviated by introducing a "gate" mechanism [47,48] and memory unit [25,49] in the LSTM network.…”

Section: Feature Extraction Based On Lstm Networkmentioning

confidence: 99%

Research on Precipitation Forecast Based on LSTM–CP Combined Model

et al. 2021

View full text Add to dashboard Cite

The tremendous progress made in the field of deep learning allows us to accurately predict precipitation and avoid major and long-term disruptions to the entire socio-economic system caused by floods. This paper presents an LSTM–CP combined model formed by the Long Short-Term Memory (LSTM) network and Chebyshev polynomial (CP) as applied to the precipitation forecast of Yibin City. Firstly, the data are fed into the LSTM network to extract the time-series features. Then, the sequence features obtained are input into the BP (Back Propagation) neural network with CP as the excitation function. Finally, the prediction results are obtained. By theoretical analysis and experimental comparison, the LSTM–CP combined model proposed in this paper has fewer parameters, shorter running time, and relatively smaller prediction error than the LSTM network. Meanwhile, compared with the SVR model, ARIMA model, and MLP model, the prediction accuracy of the LSTM–CP combination model is significantly improved, which can aid relevant departments in making disaster response measures in advance to reduce disaster losses and promote sustainable development by providing them data support.

show abstract

Multiple premises entailment recognition based on attention and gate mechanism

Cited by 6 publications

References 4 publications

Cold Chain Logistics Demand Forecasting Based on Improved BP Neural Network Model

Cold Chain Logistics Demand Forecasting Based on Improved BP Neural Network Model

Investigating the Use of Different Recurrent Neural Networks for Natural Language Inference in Arabic

Research on Precipitation Forecast Based on LSTM–CP Combined Model

Contact Info

Product

Resources

About