Paraphrase detection using LSTM networks and handcrafted features

Shahmohammadi, Hassan; Dezfoulian, Mir Hossein; Mansoorizadeh, Muharram

doi:10.1007/s11042-020-09996-y

Cited by 21 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The advantage of the PCA when applying to the preprocessing stage in machine learning is also supported by studies in other realms [56,58,70,71]. Similar to the results of the research about NLP [36,72] and other topics [53,73,74], the bi-directional RNN architecture also tends to provide more accurate results and is recommended to be utilized in the GRU runoff forecasting model. For further studies, more data filtering strategies can be tested.…”

Section: Recommendations Based On the Evaluation Resultsmentioning

confidence: 56%

Impact of Input Filtering and Architecture Selection Strategies on GRU Runoff Forecasting: A Case Study in the Wei River Basin, Shaanxi, China

Wang

Liu

Yue

et al. 2020

Water

View full text Add to dashboard Cite

A gated recurrent unit (GRU) network, which is a kind of artificial neural network (ANN), has been increasingly applied to runoff forecasting. However, knowledge about the impact of different input data filtering strategies and the implications of different architectures on the GRU runoff forecasting model’s performance is still insufficient. This study has selected the daily rainfall and runoff data from 2007 to 2014 in the Wei River basin in Shaanxi, China, and assessed six different scenarios to explore the patterns of that impact. In the scenarios, four manually-selected rainfall or runoff data combinations and principal component analysis (PCA) denoised input have been considered along with single directional and bi-directional GRU network architectures. The performance has been evaluated from the aspect of robustness to 48 various hypermeter combinations, also, optimized accuracy in one-day-ahead (T + 1) and two-day-ahead (T + 2) forecasting for the overall forecasting process and the flood peak forecasts. The results suggest that the rainfall data can enhance the robustness of the model, especially in T + 2 forecasting. Additionally, it slightly introduces noise and affects the optimized prediction accuracy in T + 1 forecasting, but significantly improves the accuracy in T + 2 forecasting. Though with relevance (R = 0.409~0.763, Grey correlation grade >0.99), the runoff data at the adjacent tributary has an adverse effect on the robustness, but can enhance the accuracy of the flood peak forecasts with a short lead time. The models with PCA denoised input has an equivalent, even better performance on the robustness and accuracy compared with the models with the well manually filtered data; though slightly reduces the time-step robustness, the bi-directional architecture can enhance the prediction accuracy. All the scenarios provide acceptable forecasting results (NSE of 0.927~0.951 for T + 1 forecasting and 0.745~0.836 for T + 2 forecasting) when the hyperparameters have already been optimized. Based on the results, recommendations have been provided for the construction of the GRU runoff forecasting model.

show abstract

Section: Recommendations Based On the Evaluation Resultsmentioning

confidence: 56%

Impact of Input Filtering and Architecture Selection Strategies on GRU Runoff Forecasting: A Case Study in the Wei River Basin, Shaanxi, China

Wang

Liu

Yue

et al. 2020

Water

View full text Add to dashboard Cite

show abstract

“…By using supervised learning, many traditional techniques were applied [10][11][12][13] or more advanced, using pre-trained models, specifically BERT and its improved models such as M-BERT, XLM-R, etc. MTL is an approach that achieves the generalization of results by using the inductive transfer method [14].…”

Section: B Paraphrase Identification Methodsmentioning

confidence: 99%

English-Vietnamese Cross-Lingual Paraphrase Identification Using MT-DNN

Anh

Thanh

et al. 2021

Eng. Technol. Appl. Sci. Res.

View full text Add to dashboard Cite

Paraphrase identification is a crucial task in natural language understanding, especially in cross-language information retrieval. Nowadays, Multi-Task Deep Neural Network (MT-DNN) has become a state-of-the-art method that brings outstanding results in paraphrase identification [1]. In this paper, our proposed method based on MT-DNN [2] to detect similarities between English and Vietnamese sentences, is proposed. We changed the shared layers of the original MT-DNN from original the BERT [3] to other pre-trained multi-language models such as M-BERT [3] or XLM-R [4] so that our model could work on cross-language (in our case, English and Vietnamese) information retrieval. We also added some tasks as improvements to gain better results. As a result, we gained 2.3% and 2.5% increase in evaluated accuracy and F1. The proposed method was also implemented on other language pairs such as English – German and English – French. With those implementations, we got a 1.0%/0.7% improvement for English – German and a 0.7%/0.5% increase for English – French.

show abstract

“…Lowercasing. This process is one of the simplest but most effective forms for preprocessing [45]. For example, the words "Jakarta" and "jakarta" have similar meanings, but they represent two different words in the vector space model when not converted to the lower case.…”

Section: )mentioning

confidence: 99%

“…This is also a widely used operation in preprocessing [3], [18], [20], [24]. A sentence often contains special characters or symbols, such as dot, colon, dash, or non-ASCII characters [45].…”

Section: )mentioning

confidence: 99%

See 1 more Smart Citation

Revisiting the challenges and surveys in text similarity matching and detection methods

2022

View full text Add to dashboard Cite

The massive amount of information from the internet has revolutionized the field of natural language processing. One of the challenges was estimating the similarity between texts. This has been an open research problem although various studies have proposed new methods over the years. This paper surveyed and traced the primary studies in the field of text similarity. The aim was to give a broad overview of existing issues, applications, and methods of text similarity research. This paper identified four issues and several applications of text similarity matching. It classified current studies based on intrinsic, extrinsic, and hybrid approaches. Then, we identified the methods and classified them into lexical-similarity, syntactic-similarity, semanticsimilarity, structural-similarity, and hybrid. Furthermore, this study also analyzed and discussed method improvement, current limitations, and open challenges on this topic for future research directions. As the results, this paper highlighted the importance of selecting the appropriate preprocessing algorithms to reduce data dimensionality and also combining several algorithms to enhance the overall matching and detection process.

show abstract

Paraphrase detection using LSTM networks and handcrafted features

Cited by 21 publications

References 22 publications

Impact of Input Filtering and Architecture Selection Strategies on GRU Runoff Forecasting: A Case Study in the Wei River Basin, Shaanxi, China

Impact of Input Filtering and Architecture Selection Strategies on GRU Runoff Forecasting: A Case Study in the Wei River Basin, Shaanxi, China

English-Vietnamese Cross-Lingual Paraphrase Identification Using MT-DNN

Revisiting the challenges and surveys in text similarity matching and detection methods

Contact Info

Product

Resources

About