Explaining query answers with explanation-ready databases

Abstract

With the increased generation and availability of big data in different domains, there is an imminent requirement for data analysis tools that are able to 'explain' the trends and anomalies obtained from this data to a range of users with different backgrounds. Wu-Madden (PVLDB 2013) and Roy-Suciu (SIGMOD 2014) recently proposed solutions that can explain interesting or unexpected answers to simple aggregate queries in terms of predicates on attributes. In this paper, we propose a generic framework that can support much richer, insightful explanations by preparing the database offline, so that top explanations can be found interactively at query time. The main idea in such explanation-ready databases is to pre-compute the effects of potential explanations (called interventions ), and efficiently re-evaluate the original query taking into account these effects. We formalize this notion and define an explanation-query that can evaluate all possible explanations simultaneously without having to run an iterative process, develop algorithms and optimizations, and evaluate our approach with experiments on real data.

Explaining Aggregates for Exploratory Analytics

“…Their origin stems from data provenance [12], but have since departed from such notions, and focus on explanations in varying contexts. One of such contexts is to provide explanations, represented as predicates, for simple query answers as in [15], [27], [31]. Explanations have been also used for interpreting outliers in both in-situ data [38] and in streaming data [7].…”

“…Scalability/efficiency is particularly important: Computing explanations is proved to be an NP-Hard problem [35] and generating them can take a long time [15], [31], [38] even with modest datasets. An exponential increase in data size implies a dramatic increase in the time to generate explanations.…”

Anomaly Detection and Explanation Discovery on Event Streams

“…In the backward pass, we aim to find an explanation for a detected anomaly. There are two parallel efforts on explanation discovery: In the database community, recent work [13,15] aims to discover explanations of query results, but is limited to results of group-by aggregate queries. In the machine learning community, sensitivity tests [11,12,14] are designed to determine the importance of each input feature.…”

“…In this position paper, we argue for the need of a new type of data stream analytics that can address anomaly detection and explanation discovery in a single, integrated system. In the literature these two topics have been addressed in isolation: while anomaly detection has been studied intensively in the data mining community [3,5,6], explanation discovery with the goal of human-readable formulas has recently received attention in the database community [13,15,18]. On the latter topic, the line of work [13,15] explains outliers for only group-by aggregate queries and finds a logical formula to describe a subset of tuples that contribute the most to the excessively high or low aggregate value of a specific group.…”

Similarity-aware query refinement for data exploration

“…The related works in [104,105,129] consider the problem of finding explanations for outliers in their query answers. An example of these outliers can be an aggregate value that is abnormally different from the rest of the values in an answer produced by a group-by SQL query.…”