Explainable Queries over Event Logs

Abstract

Added value can be extracted from event logs generated by business processes in various ways. However, although complex computations can be performed over event logs, the result of such computations is often difficult to explain; in particular, it is hard to determine what parts of an input log actually matters in the production of that result. This paper describes how an existing log processing library, called BeepBeep, can be extended in order to provide a form of provenance: individual output events produced by a query can be precisely traced back to the data elements of the log that contribute to (i.e. "explain") the result.

Synthia: a Generic and Flexible Data Structure Generator

“…Picker, Shrinkable, Testable, Resettable, Bounded and Action), which can be implemented by classes in order to generate, mutate or test input data on an arbitrary object. Case in point, Synthia has been used in recent scientific publications to simulate conference program committees [3], produce sequences of events to illustrate an event stream processor [2], generate web pages with layout faults [6], and create synthetic logs to test a monitor [10].…”

“…This means that successive calls to pick from this point on reproduce the same sequence of values as when the object was constructed. 2 Method duplicate creates a copy of the picker; this copy can either be stateless or stateful. A stateless copy amounts to creating a new instance obtained by calling the constructor with the same arguments as the original; on its side, a stateful copy places the new instance in the same internal state as the current state of the original.…”

Dynamic Program Analysis with Flexible Instrumentation and Complex Event Processing

“…BeepBeep processing units take care of buffering, synchronization between multiple streams, and numerous other lowly tasks that a user would otherwise need to implement directly each time. The use of a higher-level processing library provides additional benefits: BeepBeep calculations themselves can be abstracted further through the design of domain-specific languages; moreover, BeepBeep has explainability features, which allows it to extract elements of a long event stream that explain the output of a calculation [26].…”

Collecting and visualizing data lineage of Spark jobs

“…Scavenging instrumentation leverages generated logs and transforms these into provenance data. BeepBeep [14] and Google's data lake Goods [15] are centered on the analysis of event logs and extract coarse-grained data lineage by digesting these. They are therefore not bound to a specific processing engines (e.g.…”