The imprecisions of precision measures in process mining

Tax, Niek; Lu, Xixi; Sidorova, Natalia; Fahland, Dirk; Aalst, Wil M. P. van der

doi:10.1016/j.ipl.2018.01.013

Cited by 53 publications

(109 citation statements)

References 22 publications

Supporting

Mentioning

105

Contrasting

Unclassified

Order By: Relevance

“…This proposition captures the same idea as axiom 2 in [29], but it is more general. Axiom 2 only put this requirement on precision when τ (l) ⊆ τ (m 1 ), while PrecPro1 + also concerns the situation where this does not hold.…”

Section: A Collection Of Conformance Propositionsmentioning

confidence: 55%

“…The results presented in [29] show that ETC precision (prec K and prec L ), weighted negative event precision (prec O ) and projected precision (prec P ) violate PrecPro1 + . Additionally, all remaining measures aside from anti-alignment precision (prec Q ) and eigenvalue precision (prec R ) violate the proposition.…”

Section: Evaluation Of Existing Precision Measuresmentioning

confidence: 89%

“…Based on this proposition, rec(l, disc ofit (l)) = rec(l, disc ufit (l)) = 1 for any log l. Precision Propositions Precision (prec ∈ L × M → [0, 1]) aims to quantify the fraction of behavior allowed by the model that was actually observed. Initial work in the area of checking requirements of conformance checking measures started with [29], where five axioms for precision measures were introduced. The precision propositions that we state below partly overlap with these axioms, but some have been added and some have been strengthened.…”

Section: A Collection Of Conformance Propositionsmentioning

confidence: 99%

“…This is surprising compared to the results of our simple baseline measure. Inspired by [29], this paper takes a broad look at existing conformance measures with respect to the previously presented propositions. In the following section, existing recall and precision measures are introduced, before they will be evaluated in Section 4.6.…”

Section: Existing Recall Measuresmentioning

confidence: 99%

“…If these states show a low local precision, overall precision decreases. Furthermore, [29] showed that ETC precision (prec K and prec L ), projected precision (prec P ) and anti-alignment precision (prec Q ) also violate PrecPro2 + .…”

Section: Evaluation Of Existing Precision Measuresmentioning

confidence: 99%

See 4 more Smart Citations

Evaluating Conformance Measures in Process Mining Using Conformance Propositions

Syring

Tax

Aalst

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Process mining sheds new light on the relationship between process models and real-life processes. Process discovery can be used to learn process models from event logs. Conformance checking is concerned with quantifying the quality of a business process model in relation to event data that was logged during the execution of the business process. There exist different categories of conformance measures. Recall, also called fitness, is concerned with quantifying how much of the behavior that was observed in the event log fits the process model. Precision is concerned with quantifying how much behavior a process model allows for that was never observed in the event log. Generalization is concerned with quantifying how well a process model generalizes to behavior that is possible in the business process but was never observed in the event log. Many recall, precision, and generalization measures have been developed throughout the years, but they are often defined in an ad-hoc manner without formally defining the desired properties up front. To address these problems, we formulate 21 conformance propositions and we use these propositions to evaluate current and existing conformance measures. The goal is to trigger a discussion by clearly formulating the challenges and requirements (rather than proposing new measures). Additionally, this paper serves as an overview of the conformance checking measures that are available in the process mining area. the years, many process discovery algorithms have been developed, producing process models in various forms, such as Petri nets, process trees, and BPMN.Event logs are often incomplete, i.e., they only contain a sample of all possible behavior in the business process. This not only makes process discovery challenging; it is also difficult to assess the quality of the process model in relation to the log. Process discovery algorithms take an event log as input and aim to output a process model that satisfies certain properties, which are often referred to as the four quality dimensions [1] of process mining: (1) recall: the discovered model should allow for the behavior seen in the event log (avoiding "non-fitting" behavior), (2) precision: the discovered model should not allow for behavior completely unrelated to what was seen in the event log (avoiding "underfitting"), (3) generalization: the discovered model should generalize the example behavior seen in the event log (avoiding "overfitting"), and (4) simplicity: the discovered model should not be unnecessarily complex. The simplicity dimension refers to Occam's Razor: "one should not increase, beyond what is necessary, the number of entities required to explain anything". In the context of process mining, this is often operationalized by quantifying the complexity of the model (number of nodes, number of arcs, understandability, etc.). We do not consider the simplicity dimension in this paper, since we focus on behavior and abstract from the actual model representation. Recall is often referred to as fitness in process min...

show abstract

Section: A Collection Of Conformance Propositionsmentioning

confidence: 55%

Section: Evaluation Of Existing Precision Measuresmentioning

confidence: 89%

Section: A Collection Of Conformance Propositionsmentioning

confidence: 99%

Section: Existing Recall Measuresmentioning

confidence: 99%

Section: Evaluation Of Existing Precision Measuresmentioning

confidence: 99%

See 3 more Smart Citations