A Survey of Text Representation and Embedding Techniques in NLP

Patil, Rajvardhan; Boit, Sorio; Gudivada, Venkat N.; Nandigam, Jagadeesh

doi:10.1109/access.2023.3266377

Cited by 51 publications

(8 citation statements)

References 113 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(5) Intelligent retrieval Implement intelligent retrieval in different scenarios based on NLP technology and knowledge graph matching techniques [16]. After understanding users' retrieval intentions, conduct precise or fuzzy searches based on their query methods.…”

Section: Research Frameworkmentioning

confidence: 99%

Construction Method and Practical Application of Oil and Gas Field Surface Engineering Case Database Based on Knowledge Graph

Xia,

Dai,

Zhang

et al. 2024

Processes

View full text Add to dashboard Cite

To address the challenge of quickly and efficiently accessing relevant management experience for a wide range of ground engineering construction projects, supporting project management with information technology is crucial. This includes the establishment of a case database and an application platform for intelligent search and recommendations. The article leverages Optical Character Recognition (OCR) technology, knowledge graph technology, and Natural Language Processing (NLP) technology. It explores the mechanisms for classifying construction cases, methods for constructing a case database, structuring case data, intelligently retrieving and matching cases, and intelligent recommendation methods. This research forms a complete, feasible, and scalable method for deconstructing, storing, intelligently retrieving, and recommending construction cases, providing a theoretical basis for the establishment of a construction case database. It aims to meet the needs of digital project management and intelligent decision-making support in the oil and gas sector, thereby enhancing the efficiency and accuracy of project construction. This work offers a theoretical foundation for the development of an intelligent management platform for ground engineering projects in the oil and gas industry, supporting the sector’s digital transformation and intelligent development.

show abstract

Section: Research Frameworkmentioning

confidence: 99%

Construction Method and Practical Application of Oil and Gas Field Surface Engineering Case Database Based on Knowledge Graph

Xia,

Dai,

Zhang

et al. 2024

Processes

View full text Add to dashboard Cite

show abstract

“…TFIDF is a statistical method to evaluate the significance of a word in a document or corpus (33) . Terms with higher TF-IDF scores are more relevant and can be utilised in tasks such as keyword extraction, document ranking, and information retrieval (34) . As https://www.indjst.org/ for TFIDF, the value of IDF must be calculated first before the final calculation can be executed.…”

Section: Term Weighting and Analysismentioning

confidence: 99%

Identifying Terms to Represent Concepts of a Work Process Ontology

Osman,

Noah,

Saad

2024

IJST

View full text Add to dashboard Cite

Objectives: To identify ontology concepts from text documents for the construction of work process ontology. Methods: This study proposes a methodology to identify terms representing a work process ontology concept from a document. The methodology encompasses several key steps: document collecting, text pre-processing, term weighting and analysis, terms mapping, and domain expert relevance judgments. A comparison between the results of three different term weighting schemes, namely the Term Frequency (TF), Term Frequency-Inverse Document Frequency (TFIDF), and Mutual Information (MI) is made with the ontology concept that the domain expert has judged. Findings: The approaches adopted in this study have managed to extract ontological concepts from the targeted domain knowledge source. The findings of the comparison analysis suggest that the TFIDF term weighting scheme exhibits better results compared to the TF and MI weighting schemes. Novelty: A work process ontology is a structured knowledge describing daily operations in the government sector. However, there has been little to no effort in building the work process ontology. This study presents an integrated approach for identifying ontology concepts from documents within the domain of the work process. To the utmost extent of our understanding, this research initiative is the initial attempt to introduce a structured methodology for the semi-automatic extraction and evaluation of concepts and relationships within this domain. The findings can be utilised as a foundation for developing an ontology in the specific field. Keywords: Ontology, Work process, Text extraction, Natural language processing, Term weighting

show abstract

“…Sentence embeddings in Natural Language Processing (NLP) refer to techniques that capture the semantic meaning of entire sentences by representing them as dense numerical vectors, enabling a wide range of downstream tasks such as sentence similarity, paraphrase detection, and text classification. BERT (Bidirectional Encoder Representations from Transformers) [35][36][37] embeddings capture rich contextual information and have revolutionized NLP tasks such as text classification, named entity recognition, and sentiment analysis. BOW and TF-IDF [38] embeddings are simpler but still useful for tasks such as document classification or information retrieval, where word frequency or presence is crucial.…”

Section: Sentence Embeddingsmentioning

confidence: 99%

Reduction of Neural Machine Translation Failures by Incorporating Statistical Machine Translation

et al. 2023

View full text Add to dashboard Cite

This paper proposes a hybrid machine translation (HMT) system that improves the quality of neural machine translation (NMT) by incorporating statistical machine translation (SMT). Therefore, two NMT systems and two SMT systems were built for the Slovenian–English language pair, each for translation in one direction. We used a multilingual language model to embed the source sentence and translations into the same vector space. From each vector, we extracted features based on the distances and similarities calculated between the source sentence and the NMT translation, and between the source sentence and the SMT translation. To select the best possible translation, we used several well-known classifiers to predict which translation system generated a better translation of the source sentence. The proposed method of combining SMT and NMT in the hybrid system is novel. Our framework is language-independent and can be applied to other languages supported by the multilingual language model. Our experiment involved empirical applications. We compared the performance of the classifiers, and the results demonstrate that our proposed HMT system achieved notable improvements in the BLEU score, with an increase of 1.5 points and 10.9 points for both translation directions, respectively.

show abstract

A Survey of Text Representation and Embedding Techniques in NLP

Cited by 51 publications

References 113 publications

Construction Method and Practical Application of Oil and Gas Field Surface Engineering Case Database Based on Knowledge Graph

Construction Method and Practical Application of Oil and Gas Field Surface Engineering Case Database Based on Knowledge Graph

Identifying Terms to Represent Concepts of a Work Process Ontology

Reduction of Neural Machine Translation Failures by Incorporating Statistical Machine Translation

Contact Info

Product

Resources

About