Neural relation extraction: a survey

Aydar, Mehmet; Bozal, Özge; Ozbay, Furkan

doi:10.48550/arxiv.2007.04247

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[1]. Relation extraction is done at different levels including sentence level, bag level, and document level, using supervised models, conventional neural models, or pre-trained models [95] .…”

Section: Preprocessing and Word Embeddingsmentioning

confidence: 99%

A Review of State of the Art Deep Learning Models for Ontology Construction

Zengeya,

Vincent Fonou-Dombeu

2024

IEEE Access

View full text Add to dashboard Cite

ABSRTACT Researchers are working towards automation of ontology construction to manage the evergrowing data on the web. Currently, there is a shift from the use of machine learning techniques towards exploration of deep learning models for ontology construction. Deep learning model are capable of extracting terms, entities, relations, and classifications, and perform axiom learning from the underutilized richness of web-based knowledge. There has been remarkable progress in automatic ontology creation using deep learning models since they can perform word embedding, long-term dependency acquisition, concept extraction from large corpora, and inference of abstracted relationships based on broad corpora. Despite their emerging importance, deep learning models remain underutilized in ontology construction, and there is no comprehensive review of their application in ontology learning. This paper presents a comprehensive review of existing deep-learning models for the construction of ontologies, the strength and the weaknesses presented by the deep learning models for ontology learning as well as promising directions to achieve a more robust deep learning models. The Deep Learning models reviewed include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Long-Short Term Memory (LSTMs), and Gated Recurrent Unit (GRU) as well as their ensembles. While these traditional deep learning models have achieved great success, one of their limitations is that they struggle to understand the meaning and order of data in sequences. CNNs and RNN-based models such as LSTMs and GRUs can be computationally expensive due to their large number of parameters or complex gating mechanisms. Furthermore, RNN models suffer from vanishing gradients, making it difficult to learn long-term relationships in sequences. Additionally, RNN-based models process information sequentially, limiting their ability to take advantage of powerful parallel computing hardware, slowing down training and inference, especially for long sequences. Consequently, there has been a shift towards Generative Pre-Trained (GPT) models and Bidirectional Encoder Representations from Transformers (BERT) models. This paper also reviewed the GPT-3, GPT-4, and the BERT models for extracting terms, entities, relations, and classifications. While GPT models excel in contextual understanding and flexibility, they fall short when handling domain-specific terminology and disambiguating complex relationships. Fine-tuning and domain-specific training data could minimize these shortcomings, and further enhance the performance of GPT in term and relation extraction tasks. On the other hand, the BERT models excel in comprehending context-heavy texts, but struggles with higher-level abstraction and inference tasks due to a lack of explicit semantic knowledge, thus necessitating inference for unspecified relationships. The paper recommends further research on deep learning models for ontology alignment and merging. Also, the ensembling of deep learning models and the u...

show abstract

Section: Preprocessing and Word Embeddingsmentioning

confidence: 99%

A Review of State of the Art Deep Learning Models for Ontology Construction

Zengeya,

Vincent Fonou-Dombeu

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Relation extraction is another key part of the KG construction process; it is mainly performed to discern semantic connections between entities. Currently, the relation extraction methods mainly consist of unsupervised/semi-supervised, supervised learning, and template-based approaches [40]. Notably, supervised learning methods demonstrate superior performance.…”

Section: Educational Kgsmentioning

confidence: 99%

CourseKG: An Educational Knowledge Graph Based on Course Information for Precision Teaching

Li,

Liang,

Yang

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

With the rapid development of advanced technologies, such as artificial intelligence and deep learning, educational informatization has entered a new era. However, the explosion of information has brought numerous challenges. Knowledge graphs, as a crucial component of artificial intelligence, can contribute to the quality of teaching. This study proposes an educational knowledge graph based on course information named CourseKG for precision teaching. Precision teaching seeks to individualize the curriculum for each learner and optimize learning efficiency. CourseKG aims to establish a correct and comprehensive curriculum knowledge system and promote personalized learning paths. CourseKG can address the issue that current general-purpose knowledge graphs are not suitable for the education field. Particularly, this study proposes a framework for educational entity recognition based on the pre-trained BERT model. This framework captures relevant information in the educational domain using the BERT model and combines it with the BiGRU and multi-head self-attention mechanism to extract multi-scale and multi-level global dependency relationships. In addition, the CRF is used for character-label decoding. Further, a relationship extraction method based on the BERT model, which integrates sentence features and educational entities and estimates the similarity between knowledge pairs using cosine similarity, is proposed. The proposed CourseKG is verified by experiments using real-world C programming course data. The experimental results demonstrate the effectiveness of CourseKG. Finally, the results show that the proposed CourseKG can significantly enhance the precision teaching quality and realize multi-directional adaptation among teachers, courses, and students.

show abstract

Neural relation extraction: a survey

Cited by 2 publications

References 0 publications

A Review of State of the Art Deep Learning Models for Ontology Construction

A Review of State of the Art Deep Learning Models for Ontology Construction

CourseKG: An Educational Knowledge Graph Based on Course Information for Precision Teaching

Contact Info

Product

Resources

About