Transformer based network for Open Information Extraction

Han, Jiabao; Wang, Hongzhi

doi:10.1016/j.engappai.2021.104262

Cited by 15 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Named Entity Recognition (NER) is used to identify the location of the complaint by extracting the complaint text in Indonesian on Twitter. In recent years, the NER model has often been used in deep learning, which uses convolutional neural networks and is proven to perform better than machine learning [12]. However, the structure of a neural network needs to be trained from scratch according to speci c tasks and goals, so it takes a lot of time and resources.…”

Section: Related Workmentioning

confidence: 99%

“…Pre x I-(Inside) indicates the next word after the rst word of an entity [24], [25]. The NER model used is transformer-based because it is proven to have excellent performance due to a selfattention mechanism [12]. The NER models are BERT and XLNet, which will train tweet complaint data to study entities such as location, geographic entity, building, road measurement, natural place, time, date, object, measurement, and other entities.…”

Section: Data Annotationmentioning

confidence: 99%

“…The deep learning approach using transformer models is proven to have outstanding performance due to the self-attention mechanism. The task of extracting information is considered a sequence-to-sequence problem, and self-attention has considerable ability to calculate the representation of token positions in a sequence [12]. This research aims to build a complaint monitoring system on Indonesian texts from Twitter by extracting transformer-based information.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Transformer-based Information Extraction from Twitter Text on Complaint Monitoring System

Purwitasari,

Fatichah,

Ariyanto

et al. 2023

Preprint

View full text Add to dashboard Cite

Customer complaints receive more attention from a company because bad handling of complaints will cause the loss of existing customers. Current customers tend to use social media to report complaints by including information on the location of the complaint so that the location also needs to be extracted because it can be authentic geographic evidence in the complaint monitoring system. The number of complaint reports via social media is increasing occasionally, making the manual complaint monitoring system very inefficient because it takes a long time. This study proposes an automatic complaint monitoring system on Indonesian text from Twitter by extracting transformer-based information. The automatic complaint monitoring system uses a combination of the Bidirectional Encoder Representation from Transformer (BERT) model to extract location information on complaint tweets and the Convolutional Neural Network (CNN) model to classify the complaint type, which gets the highest F1 score of 0.90471. The complaint monitoring system is also visualized in the complaint locations to make it easier for companies to find the location of complaint reports, and action can be taken to deal with public complaints immediately.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Data Annotationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transformer-based Information Extraction from Twitter Text on Complaint Monitoring System

Purwitasari,

Fatichah,

Ariyanto

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent innovations in deep neural networks have led to impressive advances in Natural Language Processing (NLP) (Devlin et al, 2018;Wolf et al, 2020). These advances include new stateof-the-art results in tasks as diverse as question answering, information extraction, sentiment analysis, conversational 'chatbot' agents and summarization, to only name a few (Reddy et al, 2019;Han and Wang, 2021;Naseem et al, 2020;Siblini et al, 2019;Liu, 2019). Due to the performance of these models, it has also become possible in recent years to use NLP tools for computational social science and digital humanities.…”

Section: Introductionmentioning

confidence: 99%

Quantifying Gender Disparity in Pre-Modern English Literature using Natural Language Processing

Kejriwal¹,

Nagaraj²

2023

Journal of Data Science

View full text Add to dashboard Cite

Research has continued to shed light on the extent and significance of gender disparity in social, cultural and economic spheres. More recently, computational tools from the data science and Natural Language Processing (NLP) communities have been proposed for measuring such disparity at scale using empirically rigorous methodologies. In this article, we contribute to this line of research by studying gender disparity in 2,443 copyright-expired literary texts published in the pre-modern period, defined in this work as the period ranging from the beginning of the nineteenth through the early twentieth century. Using a replicable data science methodology relying on publicly available and established NLP components, we extract three different gendered character prevalence measures within these texts. We use an extensive set of statistical tests to robustly demonstrate a significant disparity between the prevalence of female characters and male characters in pre-modern literature. We also show that the proportion of female characters in literary texts significantly increases in female-authored texts compared to the same proportion in male-authored texts. However, regression-based analysis shows that, over the 120 year period covered by the corpus, female character prevalence does not change significantly over time, and remains below the parity level of 50%, regardless of the gender of the author. Qualitative analyses further show that descriptions associated with female characters across the corpus are markedly different (and stereotypical) from the descriptions associated with male characters.

show abstract

“…However, most transformerbased models have a quadratic complexity, which limits the token length as a trade-off between performance and memory usage, resulting in truncating training sentences (Fan et al, 2020). The authors of (Han and Wang, 2021) introduced a transformer-based OIE model, however, its performance was not evaluated against any state-of-the-art neural network model. Thus, as a future direction, we intend to further evaluate different neural OIE research trends, including transformer-based models on benchmark datasets.…”

Section: Results and Evaluationmentioning

confidence: 99%

Open Information Extraction for Knowledge Representation

Sarhan¹

View full text Add to dashboard Cite

The field of Natural Language Processing (NLP) focuses on developing computational techniques to analyze and extract information from human language. With the exponential growth of unstructured textual data, NLP-based techniques have become essential for extracting valuable insights from this data. However, existing information extraction systems have limitations in terms of extracting valuable information without predefined relations or ontology and storing the extracted knowledge effectively. This Ph.D. thesis aims to enhance open information extraction methods to represent unstructured textual data efficiently and effectively. The first part of the research focuses on Open Information Extraction (OIE) systems and their challenges. Existing OIE methods, including pattern-based and machine learning-based approaches, as well as neural techniques, are analyzed to understand their limitations. A Bidirectional Gated Recurrent Unit (Bi-GRU) OIE model is proposed in Chapter 3, which utilizes contextualized word embeddings to extract relevant triples from unstructured text. Experimental results demonstrate the effectiveness of this model in generating high-quality relation triples. Chapter 4 addresses the lack of labeled data, a common problem in NLP tasks. The research extends the OIE model from Chapter 3 by using learned features to generate relation triples and explores the transferability of these features across different OIE domains and the related task of Relation Extraction (RE). The results show comparable performance with traditional training, indicating the potential of OIE in achieving NLP performance without labeled data. In Chapter 5, the focus shifts to enhancing pre-trained language models for taxonomy classification. Pre-trained language models often struggle with unseen patterns during inference, and the limited size of annotated data poses a challenge. A two-stage fine-tuning procedure, incorporating data augmentation techniques, is proposed to improve the generalizability of pre-trained models. Experimental results demonstrate strong generalizability on unseen data, with an F1 score of 91.25%. Chapter 6 explores the use of OIE for constructing a knowledge graph, specifically in the context of cyber threat intelligence. Open-CyKG, an open cyber threat intelligence knowledge graph framework, is designed using an attention-based neural OIE model and a Named Entity Recognition (NER) model. Refinement and canonicalization techniques are employed to overcome ambiguity and data redundancy during knowledge graph construction. The results show that querying the constructed knowledge graph can be done efficiently, highlighting the support of OIE in knowledge graph development. The proposed components achieve beyond-state-of-the-art results in terms of OIE performance, NER performance, and knowledge graph canonicalization. The research presented in the previous chapters demonstrates significant improvements in the efficiency and effectiveness of open information extraction methods for representing unstructured textual data. These advancements leverage techniques such as data augmentation, multi-stage fine-tuning, and pre-trained language models. The construction of knowledge graphs, enabled by OIE, has the potential to mimic human intelligence and benefit various complex applications, including recommender systems, search engines, and dialog systems.

show abstract

Transformer based network for Open Information Extraction

Cited by 15 publications

References 7 publications

Transformer-based Information Extraction from Twitter Text on Complaint Monitoring System

Transformer-based Information Extraction from Twitter Text on Complaint Monitoring System

Quantifying Gender Disparity in Pre-Modern English Literature using Natural Language Processing

Open Information Extraction for Knowledge Representation

Contact Info

Product

Resources

About