BertNet: Harvesting Knowledge Graphs from Pretrained Language Models

Shibo, Hao,; Tan, Bowen; Tang, Kaiwen; Zhang, Hengzhe; Xing, Eric P.; Hu, Zhiting

doi:10.48550/arxiv.2206.14268

Cited by 5 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PLMs have been used to extract KGs. Hao et al (2022) use BERT-like models to extract knowledge of arbitrary new relation types and entities, without being restricted by preexisting knowledge or corpora, evaluating the outcomes using human effort. Wang et al (Wang et al, 2020) proceed similarly but compare the quality with existing KGs, which is an approach closer to the experiment reported here.…”

Section: Large Language Models and Knowledge Graphsmentioning

confidence: 99%

Enhancing knowledge graphs with microdata and LLMs: the case of Schema.org and Wikidata in touristic information

Gonzalez-Garcia,

González-Carreño,

Rivas Machota

et al. 2024

View full text Add to dashboard Cite

Purpose Knowledge graphs (KGs) are structured knowledge bases that represent real-world entities and are used in a variety of applications. Many of them are created and curated from a combination of automated and manual processes. Microdata embedded in Web pages for purposes of facilitating indexing and search engine optimization are a potential source to augment KGs under some assumptions of complementarity and quality that have not been thoroughly explored to date. In that direction, this paper aims to report results on a study that evaluates the potential of using microdata extracted from the Web to augment the large, open and manually curated Wikidata KG for the domain of touristic information. As large corpora of Web text is currently being leveraged via large language models (LLMs), these are used to compare the effectiveness of the microdata enhancement method. Design/methodology/approach The Schema.org taxonomy was used as the source to determine the annotation types to be collected. Here, the authors focused on tourism-related pages as a case study, selecting the relevant Schema.org concepts as point of departure. The large CommonCrawl resource was used to select those annotations from a large recent sample of the World Wide Web. The extracted annotations were processed and matched with Wikidata to estimate the degree to which microdata produced for SEO might become a valuable resource to complement KGs or vice versa. The Web pages themselves can also serve as a context to produce additional metadata elements using them as context in pipelines of an existing LLMs. That way, both the annotations and the contents itself can be used as sources. Findings The samples extracted revealed a concentration of metadata annotations in only a few of the relevant Schema.org attributes and also revealed the possible influence of authoring tools in a significant fraction of microdata produced. The analysis of the overlapping of attributes in the sample with those of Wikidata showed the potential of the technique, limited by the disbalance of the presence of attributes. The combination of those with the use of LLMs to produce additional annotations demonstrates the feasibility of the approach in the population of existing Wikidata locations. However, in both cases, the effectiveness appears to be lower in the cases of less content in the KG, which are arguably the most relevant when considering the scenario of an automated population approach. Originality/value The research reports novel empirical findings on the way touristic annotations with a SEO orientation are being produced in the wild and provides an assessment of their potential to complement KGs, or reuse information from those graphs. It also provides insights on the potential of using LLMs for the task.

show abstract

Section: Large Language Models and Knowledge Graphsmentioning

confidence: 99%

Enhancing knowledge graphs with microdata and LLMs: the case of Schema.org and Wikidata in touristic information

Gonzalez-Garcia,

González-Carreño,

Rivas Machota

et al. 2024

View full text Add to dashboard Cite

show abstract

“…Many approaches have been proposed to achieve this link prediction task, some focused on observable features such as Rule Mining [17] [16][38] [24] or the Path Ranking Algorithm [31] [32], and others focused on capturing latent features of the graph by using different embedding techniques. In our paper, we are mainly focusing on the KG embedding approaches.…”

Section: Knowledge Graph Embedding and Link Predictionmentioning

confidence: 99%

“…BertNet [24] tried to address this issue along with the dependency of having existing massive data to learn from. They proposed to apply a paraphrasing stage before extracting the triplets, that way, there would be a more diverse set of alternatives to generate the entities and the triplets from.…”

Section: Knowledge Graph Constructionmentioning

confidence: 99%

Can Incremental Learning help with KG Completion?

Osama¹,

Elkheir²

2023

Computer Science, Engineering and Applications

View full text Add to dashboard Cite

Knowledge Graphs (KGs) are a type of knowledge representation that gained a lot of attention due to their ability to store information in a structured format. This structure representation makes KGs naturally suited for search engines and NLP tasks like question-answering (QA) and task-oriented systems; however, KGs are hard to construct. While QA datasets are more available and easier to construct, they lack structural representation. This availability of QA datasets made them a rich resource for machine learning models, but these models benefit from the implicit structure in such datasets. We propose a framework to make this structure more pronounced and extract KG from QA datasets in an end-to-end manner, allowing the system to learn new knowledge in incremental learning with a human-in-the-loop (HITL) when needed. We test our framework using the SQuAD dataset and our incremental learning approach with two datasets, YAGO3-10 and FB15K237, both of which show promising results.

show abstract

“…Prior work showed that prompts can be automatically optimized to produce factually correct claims more robustly (Lester et al, 2021;Zhong et al, 2021;Qin and Eisner, 2021). Hao et al (2022) utilized multiple generated paraphrases to gauge consistency (Hao et al, 2022), and other works (Elazar et al, 2021; further proposed training objectives to improve model consistency. Another approach to handling multiple outputs is via variants of decoding strategies , or model ensembles (Sun et al, 2022).…”

Section: Model Calibrationmentioning

confidence: 99%

LM vs LM: Detecting Factual Errors via Cross Examination

Cohen,

Hamri,

Geva

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically. Inspired by truth-seeking mechanisms in law, we propose a factuality evaluation framework for LMs that is based on cross-examination. Our key idea is that an incorrect claim is likely to result in inconsistency with other claims that the model generates. To discover such inconsistencies, we facilitate a multi-turn interaction between the LM that generated the claim and another LM (acting as an examiner) which introduces questions to discover inconsistencies. We empirically evaluate our method on factual claims made by multiple recent LMs on four benchmarks, finding that it outperforms existing methods and baselines, often by a large gap. Our results demonstrate the potential of using interacting LMs to capture factual errors.

show abstract

BertNet: Harvesting Knowledge Graphs from Pretrained Language Models

Cited by 5 publications

References 0 publications

Enhancing knowledge graphs with microdata and LLMs: the case of Schema.org and Wikidata in touristic information

Enhancing knowledge graphs with microdata and LLMs: the case of Schema.org and Wikidata in touristic information

Can Incremental Learning help with KG Completion?

LM vs LM: Detecting Factual Errors via Cross Examination

Contact Info

Product

Resources

About