Knowledge expansion over probabilistic knowledge bases

Abstract

Information extraction and human collaboration techniques are widely applied in the construction of web-scale knowledge bases. However, these knowledge bases are often incomplete or uncertain. In this paper, we present ProbKB, a probabilistic knowledge base designed to infer missing facts in a scalable, probabilistic, and principled manner using a relational DBMS. The novel contributions we make to achieve scalability and high quality are: 1) We present a formal definition and a novel relational model for probabilistic knowledge bases. This model allows an efficient SQL-based inference algorithm for knowledge expansion that applies inference rules in batches; 2) We implement ProbKB on massive parallel processing databases to achieve further scalability; and 3) We combine several quality control methods that identify erroneous rules, facts, and ambiguous entities to improve the precision of inferred facts. Our experiments show that ProbKB system outperforms the state-of-the-art inference engine in terms of both performance and quality.

Incremental knowledge base construction using DeepDive

“…We formalized the development process and built DeepDive to ease and accelerate the KBC process, which we hope is of interest to many of these systems as well. Deep-Dive has many common features to Chen and Wang [11], Google's Knowledge Vault [14], and a forerunner of Deep-Dive, Tuffy [30]. We focus on the incremental evaluation from feature extraction to inference.…”

“…The process of populating a structured relational database from unstructured sources has received renewed interest in the database community through high-profile start-up companies (e.g., Tamr and Trifacta), established companies like IBM's Watson [7,16], and a variety of research efforts [11,25,28,36,40]. At the same time, communities such as natural language processing and machine learning are attacking similar problems under the name knowledge base construction (KBC) [5,14,23].…”

“…1 DeepDive's language and execution model are similar to other KBC systems: DeepDive uses a high-level declarative language [11,28,30]. From a database perspective, DeepDive's language is based on SQL.…”

“…From a machine learning perspective, DeepDive's language is based on Markov Logic [13,30]: DeepDive's language inherits Markov Logic Networks' (MLN's) formal semantics. 2 Moreover, it uses a standard execution model for such systems [11,28,30] in which programs go through two main phases: grounding, in which one evaluates a sequence of SQL queries to produce a data structure called a factor graph that describes a set of random variables and how they are correlated. Essentially, every tuple in the database or result of a query is a random variable (node) in this factor graph.…”

Ontological Pathfinding

“…These knowledge bases store structured information about real-world people, places, organizations, etc. They are constructed by human crafting (DBPedia, Freebase), information extraction (DeepDive, OpenIE, ProBase), reasoning and inference [12,46], knowledge fusion [52,16], or combinations of them (NELL).…”