Lexical knowledge enhanced text matching via distilled word sense disambiguation

Pu, Xiaorong; Ye, Lin; Leng, Jiaxu; Wu, Tao; Gao, Xinbo

doi:10.1016/j.knosys.2023.110282

Cited by 7 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each synset is described by its definition, surface forms (lemmas), examples of usage (where available), and the relations between synsets, e.g., hypernymy (is-a), meronymy (is-part) or troponymy (manner-of). WN's primary use in NLP is as a sense inventory (Agirre and Edmonds, 2007;Pu et al, 2023).…”

Section: Datasetsmentioning

confidence: 99%

3D-EX: A Unified Dataset of Definitions and Dictionary Examples

Almeman,

Sheikhi,

Espinosa-Anke

2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

Definitions are a fundamental building block in lexicography, linguistics and computational semantics. In NLP, they have been used for retrofitting word embeddings or augmenting contextual representations in language models. However, lexical resources containing definitions exhibit a wide range of properties, which has implications in the behaviour of models trained and evaluated on them. In this paper, we introduce 3D-EX, a dataset that aims to fill this gap by combining well-known English resources into one centralized knowledge repository in the form of triples. 3D-EX is a unified evaluation framework with carefully pre-computed train/validation/test splits to prevent memorization. We report experimental results that suggest that this dataset could be effectively leveraged in downstream NLP tasks. Code and data are available at https://github.com/ F-Almeman/3D-EX.

show abstract

Section: Datasetsmentioning

confidence: 99%

3D-EX: A Unified Dataset of Definitions and Dictionary Examples

Almeman,

Sheikhi,

Espinosa-Anke

2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

show abstract

“…Literature [16] suggested a method to enhance text match by integrating lexical knowledge from external resources and simulating the semantics of potentially ambiguous words. Through simulation, it has been corroborated that this method greatly improves the text-matching performance of a Chinese corpus.…”

Section: Introductionmentioning

confidence: 99%

A Study on Lexical Disambiguation in English Translation Based on Twin Neural Networks

Cui

2024

Applied Mathematics and Nonlinear Sciences

View full text Add to dashboard Cite

To solve the problem of the lack of effective algorithmic models to improve the accuracy of lexical disambiguation in English translation, this paper constructs a twin network lexical disambiguation model based on the characteristics of twin networks, and studies the construction process from the original corpus to the input sample pairs. The Stacked-LSTM algorithm is utilized to align the input Chinese and English corpus and expand the dataset. To achieve disambiguation, the input sample similarity is calculated after training the twin neural network, which extracts corpus features using BiLSTM Attention. After comparing the disambiguation experiments of various algorithms, the model of this algorithm can effectively calculate the similarity of the input samples and achieve the disambiguation accuracy of 68.23% for English vocabulary translation, and 87.0% for vocabulary segmentation of complex English sentences or articles. This shows that the model of this algorithm has good performance for disambiguating English translations.

show abstract

“…In 2021, Barba et al [16] introduced a transformer-based neural architecture that reformulated WSD as a span extraction problem, attaining a wonderful performance on the English WSD task. In 2023, Pu et al [17] proposed a lightweight WSD model, which uses the vocabulary knowledge to extract a BERTbased pretraining model. These research results in remarkable gains in model development efficiency and bringing significant performance improvements in NLP.…”

Section: Introductionmentioning

confidence: 99%

Pretrained Quantum-Inspired Deep Neural Network for Natural Language Processing

Shi,

Chen,

Lai

et al. 2024

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Natural language processing (NLP) may face the inexplicable "black-box" problem of parameters and unreasonable modeling for lack of embedding of some characteristics of natural language, while the quantum-inspired models based on quantum theory may provide a potential solution. However, the essential prior knowledge and pretrained text features are often ignored at the early stage of the development of quantum-inspired models. To attacking the above challenges, a pretrained quantuminspired deep neural network is proposed in this work, which is constructed based on quantum theory for carrying out strong performance and great interpretability in related NLP fields. Concretely, a quantum-inspired pretrained feature embedding (QPFE) method is first developed to model superposition states for words to embed more textual features. Then, a QPFE-ERNIE model is designed by merging the semantic features learned from the prevalent pretrained model ERNIE, which is verified with two NLP downstream tasks: 1) sentiment classification and 2) word sense disambiguation (WSD). In addition, schematic quantum circuit diagrams are provided, which has potential impetus for the future realization of quantum NLP with quantum device. Finally, the experiment results demonstrate QPFE-ERNIE is significantly better for sentiment classification than gated recurrent unit (GRU), BiLSTM, and TextCNN on five datasets in all metrics and achieves better results than ERNIE in accuracy, F1-score, and precision on two datasets (CR and SST), and it also has advantage for WSD over the classical models, including BERT (improves F1-score by 5.2 on average) and ERNIE (improves F1-score by 4.2 on average) and improves the F1-score by 8.7 on average compared with a previous quantum-inspired model QWSD. QPFE-ERNIE provides a novel pretrained quantuminspired model for solving NLP problems, and it lays a foundation for exploring more quantum-inspired models in the future.

show abstract

Lexical knowledge enhanced text matching via distilled word sense disambiguation

Cited by 7 publications

References 42 publications

3D-EX: A Unified Dataset of Definitions and Dictionary Examples

3D-EX: A Unified Dataset of Definitions and Dictionary Examples

A Study on Lexical Disambiguation in English Translation Based on Twin Neural Networks

Pretrained Quantum-Inspired Deep Neural Network for Natural Language Processing

Contact Info

Product

Resources

About