TensorLog: A Probabilistic Database Implemented Using Deep-Learning Infrastructure

Abstract

We present an implementation of a probabilistic first-order logic called TensorLog, in which classes of logical queries are compiled into differentiable functions in a neural-network infrastructure such as Tensorflow or Theano. This leads to a close integration of probabilistic logical reasoning with deep-learning infrastructure: in particular, it enables high-performance deep learning frameworks to be used for tuning the parameters of a probabilistic logic. The integration with these frameworks enables use of GPU-based parallel processors for inference and learning, making TensorLog the first highly parallellizable probabilistic logic. Experimental results show that TensorLog scales to problems involving hundreds of thousands of knowledge-base triples and tens of thousands of examples.

Modeling Content and Context with Deep Relational Learning

“…These systems identify classes of logical queries that can be compiled into differentiable functions in a neural network infrastructure. In this space we have Tensor Logic Networks (TLNs) (Donadello et al, 2017) and TensorLog (Cohen et al, 2020). Symbols are represented as row vectors in a parameter matrix.…”

“…Probabilistic Logics: We compare to PSL (Bach et al, 2017), a purely symbolic probabilistic logic, and TensorLog (Cohen et al, 2020), a neuro-symbolic one. In both cases, we instantiate the program using the weights learned with our base encoders.…”

“…Motivated by these challenges, we introduce DRAIL 1 , a Deep Relational Learning framework, which uses a combined neuro-symbolic representation for modeling the interaction between multiple decisions in relational domains. Similar to other neuro-symbolic approaches Cohen et al, 2020), our goal is to exploit the complementary strengths of the two modeling paradigms. Symbolic representations, used by logic-based systems and by probabilistic graphical models (Richardson and Domingos, 2006;Bach et al, 2017), are interpretable, and allow domain experts to directly inject knowledge and constrain the learning problem.…”

“…Our main design goal in DRAIL is to provide a generalized tool, specifically designed for NLP tasks. Existing approaches designed for classic relational learning tasks (Cohen et al, 2020), such as knowledge graph completion, are not equipped to deal with the complex linguistic input, whereas others are designed for very specific NLP settings such as word-based quantitative reasoning problems (Manhaeve et al, 2018) or aligning images with text . We discuss the differences between DRAIL and these approaches in Section 2.…”

Deeply Embedded Knowledge Representation & Reasoning For Natural Language Question Answering: A Practitioner’s Perspective

“…Our work is related to all the works in Neuro-Symbolic reasoning (Serafini and Garcez, 2016;Cohen et al, 2020;Rocktäschel and Riedel, 2017;Kazemi and Poole, 2018;Aspis et al, 2018;Ebrahimi et al, 2018;Evans and Grefenstette, 2018) that aims at implementing a symbolic theorem prover with Neural Networks. These works provides proof that more complicated symbolic reasoning algorithms than the one used in this work, can be implemented using neural nets.…”