On the Understandability of Temporal Properties Formalized in Linear Temporal Logic, Property Specification Patterns and Event Processing Language

Abstract

Temporal properties are important in a wide variety of domains for different purposes. For example, they can be used to avoid architectural drift in software engineering or to support the regulatory compliance of business processes. In this work, we study the understandability of three major temporal property representations: (1) Linear Temporal Logic (LTL) is a formal and well-established logic that offers temporal operators to describe temporal properties; (2) Property Specification Patterns (PSP) are a collection of recurring temporal properties that abstract underlying formal and technical representations; (3) Event Processing Language (EPL) can be used for runtime monitoring of event streams using Complex Event Processing. We conducted two controlled experiments with 216 participants in total to study the understandability of those approaches using a completely randomized design with one alternative per experimental unit. We hypothesized that PSP, as a highly abstracting pattern language, is easier to understand than LTL and EPL, and that EPL, due to separation of concerns (as one or more queries can be used to explicitly define the truth value change that an observed event pattern causes), is easier to understand than LTL. We found evidence supporting our hypotheses which was statistically significant and reproducible.

Modeling compliance specifications in linear temporal logic, event processing language and property specification patterns: a controlled experiment on understandability

“…Moreover, the results are overall in line with two controlled experiments on the understandability of already existing formal specifications in LTL, EPL and PSP carried out by Czepa and Zdun [17]. The results of these controlled experiments with 216 participants in total suggested that existing specifications in PSP are significantly easier to understand than existing specifications in EPL and LTL.…”

“…Moreover, the results are overall in line with two controlled experiments with 216 participants in total on the understandability of already existing formal specifications in LTL, EPL and PSP (cf. Czepa and Zdun [16]). In contrast to the current study, which focuses on the formal modeling of compliance specifications, no major differences between novice and moderately advanced users were found in understandability of existing specifications.…”

“…In previous controlled experiments carried out by Czepa and Zdun [17], the understandability of already existing formal specifications in those language was studied. That experiments can be seen as the first step toward studying the understandability of those languages.…”

How Understandable Are Pattern-based Behavioral Constraints for Novice Software Designers?

“…From a prior experiment on the understandability of textual behavioral constraint approaches (cf. Czepa & Zdun [16]), it is evident that higher correctness values (about 70% on the average in PSP) are achievable if access to learning material and other material (e.g., hand-written notes) is granted during the experiment session. That is, it appears to be difficult to deduce the meaning of patternbased behavioral constraints from their textual and/or graphical representations without additional support.…”

“…Section 8 which discusses those related works in depth) yields contradictory results which indicates that the understandability of pattern-based graphical and textual behavioral constraints is not yet well understood. Two prior empirical studies (both reported in [16]) indicated that the pattern-based PSP representation provides a high level of understandability (about 70 percent on average in the specific setup of those studies), but these studies did not consider graphical pattern-based behavioral constraints. As it is our experience from multiple industry projects that industry experts in areas such as business process management tend to prefer graphical over textual constraint representations when given the choice, it would be important to test if their gut feeling can be empirically confirmed.…”