Hinge-Loss Markov Random Fields and Probabilistic Soft Logic

Abstract

A fundamental challenge in developing high-impact machine learning technologies is balancing the need to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge graphs and the Web, to images, video, and natural language. In this paper, we introduce two new formalisms for modeling structured data, and show that they can both capture rich structure and scale to big data. The first, hingeloss Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model that generalizes different approaches to convex inference. We unite three approaches from the randomized algorithms, probabilistic graphical models, and fuzzy logic communities, showing that all three lead to the same inference objective. We then define HL-MRFs by generalizing this unified objective. The second new formalism, probabilistic soft logic (PSL), is a probabilistic programming language that makes HL-MRFs easy to define using a syntax based on first-order logic. We introduce an algorithm for inferring most-probable variable assignments (MAP inference) that is much more scalable than general-purpose convex optimization methods, because it uses message passing to take advantage of sparse dependency structures. We then show how to learn the parameters of HL-MRFs. The learned HL-MRFs are as accurate as analogous discrete models, but much more scalable. Together, these algorithms enable HL-MRFs and PSL to model rich, structured data at scales not previously possible.

From LSAT: The Progress and Challenges of Complex Reasoning

“…Advanced methods to solve complex reasoning problems, no matter what specific reasoning skill is required, can be summarized as following three types: symbolic models, neural models and neural-symbolic models [15,16,17].…”

“…Existing methods for complex reasoning can be summarized into three types, namely, symbolic models, neural models, and neural-symbolic models [15,16]. Symbolic models identify the discrete symbols (like entities and logical functions) as basic reasoning units, and perform explicit inferences upon symbolic representations.…”

Integrating Learning and Reasoning with Deep Logic Models

“…The structure of these latter cliques follows a template determined by the rule, that is repeated for the single groundings. The graphical model is similar to the ones built by Probabilistic Soft Logic [1] or Markov Logic Networks [26], but enriched with the nodes corresponding to the output of the neural networks.…”

LYRICS: a General Interface Layer to Integrate Logic Inference and Deep Learning

“…Markov Logic Networks (MLN) [20] and Probabilistic Soft Logic (PSL) [13,2] provide a generic AI interface layer for machine learning by implementing a probabilistic logic. However, the integration with the underlying learning processes working on the low-level sensorial data is shallow: a low-level learner can be trained independently, then frozen and stacked with the AI layer providing a higher-level inference mechanism.…”

“…Here, we assume that the patterns are represented as R 2 datapoints. The classification task is a multi-label problem where the patterns belongs to three classes A, B, C. In particular, the class assignments are defined by the following membership regions: 2]. These regions correspond to three overlapping rectangles as shown in Figure 3(a).…”