Determining Process Model Precision and Generalization with Weighted Artificial Negative Events

Abstract

Process mining encompasses the research area which is concerned with knowledge discovery from event logs. One common process mining task focuses on conformance checking, comparing discovered or designed process models with actual real-life behavior as captured in event logs in order to assess the "goodness" of the process model. This paper introduces a novel conformance checking method to measure how well a process model performs in terms of precision and generalization with respect to the actual executions of a process as recorded in an event log. Our approach differs from related work in the sense that we apply the concept of so-called weighted artificial negative events towards conformance checking, leading to more robust results, especially when dealing with less complete event logs that only contain a subset of all possible process execution behavior. In addition, our technique offers a novel way to estimate a process model's ability to generalize. Existing literature has focused mainly on the fitness (recall) and precision (appropriateness) of process models, whereas generalization has been much more difficult to estimate. The described algorithms are implemented in a number of ProM plugins, and a Petri net conformance checking tool was developed to inspect process model conformance in a visual manner.

Anti-alignments in Conformance Checking – The Dark Side of Process Models

“…Less related is the work in [17], where the introduction of weighted artificial negative events from a log is proposed. Given a log L, an artificial negative event is a trace σ = σ · a where σ ∈ L, but σ / ∈ L. Algorithms are proposed to weight the confidence of an artificial negative event, and they can be used to estimate the precision and generalization of a process model [17].…”

“…Given a log L, an artificial negative event is a trace σ = σ · a where σ ∈ L, but σ / ∈ L. Algorithms are proposed to weight the confidence of an artificial negative event, and they can be used to estimate the precision and generalization of a process model [17]. Like in [4], by only considering one step ahead of log/model's behavior, this technique may not catch serious precision/generalization problems.…”

Event-Based Real-Time Decomposed Conformance Analysis

“…First of all, we note that we apply a heuristic, event-granular replayer similar to the one applied in [15]. The reasoning behind the choice to opt for a replayer playing the token game instead of an alignment-based replayer [2] are twofold.…”

“…Note that in extreme edge cases it is possible that the forced ring of a non-enabled transition should be preferred if this avoids several other violations later in the event trace [17]. In [15], a replay strategy is proposed which prefers the ring of enabled transition mapped to the activity at hand rst, followed by the set of silent transitions, followed by the set of nonenabled transition mapped to the activity at hand. If the chosen set contains multiple transition candidates, a one-step look-ahead procedure is executed to determine which candidate enables the execution of the following activity (if no such candidate can be found, a random one is chosen).…”

“…To benchmark the performance of our developed real-time conformance analysis technique against related approaches, a tting event log was generated (based on We compare our proposed technique against the alignment based replay technique by Adriansyah et al [2] as well our original implementation of the tokengame based heuristic replayer [15]. Both the non-decomposed and decomposed variants of these techniques were included, applying hereto the methodology as described in [8].…”