Fault Localization for Declarative Models in Alloy

Wang, Kaiyuan; Sullivan, Allison; Marinov, Darko; Khurshid, Sarfraz

doi:10.1109/issre5003.2020.00044

Cited by 20 publications

(17 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluated FLACK on a benchmark consisting of a suite of buggy models from AlloyFL [19]. The experimental results corroborate that FLACK is able to consistently rank buggy expressions in the top 1.9% of the suspicious list.…”

mentioning

confidence: 59%

“…To motivate the research and illustrate our approach, we provide an Alloy specification of a finite state machine (FSM), adapted from AlloyFL benchmarks [19]. The specification defines two type signatures, i.e., S t a t e and FSM , along with their fields (lines 1-5).…”

Section: Il L U S T R a T Io Nmentioning

confidence: 99%

“…However, there is a dearth of fault localization techniques developed for Alloy. AlloyFL [19] is perhaps the only fault localization tool available for Alloy as of today. The key idea of AlloyFL is to use "unit tests," where a test is a predicate that describes an Alloy instance to encode expected behaviors, to compute suspicious expressions in an Alloy model that fails these tests.…”

mentioning

confidence: 99%

“…Indeed, instead of writing test cases, Alloy users write assertions to describe the desired property and let the Alloy Analyzer search for potential counterexamples (cex's) that violate the property. Moreover, it is unclear how many test cases are needed or how good they must be for AlloyFL to be effective (e.g., in the AlloyFL evaluation [19], the number of tests ranges from 30 to 120).…”

mentioning

confidence: 99%

See 3 more Smart Citations

FLACK: Counterexample-Guided Fault Localization for Alloy Models

Zheng

Nguyen

Brida

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Fault localization is a practical research topic that helps developers identify code locations that might cause bugs in a program. Most existing fault localization techniques are designed for imperative programs (e.g., C and Java) and rely on analyzing correct and incorrect executions of the program to identify suspicious statements. In this work, we introduce a fault localization approach for models written in a declarative language, where the models are not "executed," but rather converted into a logical formula and solved using backend constraint solvers. We present FLACK, a tool that takes as input an Alloy model consisting of some violated assertion and returns a ranked list of suspicious expressions contributing to the assertion violation. The key idea is to analyze the differences between counterexamples, i.e., instances of the model that do not satisfy the assertion, and instances that do satisfy the assertion to find suspicious expressions in the input model. The experimental results show that FLACK is efficient (can handle complex, realworld Alloy models with thousand lines of code within 5 seconds), accurate (can consistently rank buggy expressions in the top 1.9% of the suspicious list), and useful (can often narrow down the error to the exact location within the suspicious expressions). I. In t r o d u c t io nDeclarative specification languages and the corresponding formally precise analysis engines have long been utilized to solve various software engineering problems. The Alloy specification language [1 ] relies on first-order relational logic, and has been used in a wide range of applications, such as program verification [2], test case generation [3], [4], software design [5], [6 ], [7], network security [8 ], [9], [10], security analysis of emerging platforms, such as IoT [11] and Android [12], [13], and design tradeoff analysis [14], [15], to name a few. Cunha and Macedo, among others, use a recent extension of Alloy, called Electrum [16], to validate the European Rail Traffic Management System, a system of standards for management and inter-operation of signaling for the European railways [17]. Kim [18] proposes a Secure Swarm Toolkit (SST), a platform for building an authorization service infrastructure for IoT systems, and uses Alloy to show that SST provides necessary security guarantees.Similar to developing programs in an imperative language, such as C or Java, developers can make subtle mistakes when using Alloy in modeling system specifications, espe-

show abstract

mentioning

confidence: 59%

Section: Il L U S T R a T Io Nmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

FLACK: Counterexample-Guided Fault Localization for Alloy Models

Zheng

Nguyen

Brida

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

show abstract

“…Alloy is a wellknown modeling languages which comes with an automatic SATbased analyzer [13] that performs analysis using a bounded scope on the universe of discourse. Over the past few years, many Alloy extensions have been developed [3,4,14,17].…”

Section: Introductionmentioning

confidence: 99%

ASketch: a sketching framework for Alloy

Wang

Sullivan

Marinov

et al. 2018

Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

Self Cite

View full text Add to dashboard Cite

Alloy is a declarative modeling language that supports rst-order logic with transitive closure. Alloy has been used in a variety of domains to model software systems and nd design de ciencies. However, it is often challenging to make an Alloy model correct or to debug a faulty Alloy model. ASketch is a sketching/synthesis technique that can help users write correct Alloy models. ASketch allows users to provide a partial Alloy model with holes, a generator that speci es candidate fragments to be considered for each hole, and a set of tests that capture the desired model properties. Then, the tool completes the holes such that all tests for the completed model pass. ASketch uses tests written for the recently introduced AUnit framework, which provides a foundation of testing (unit tests, test execution, and model coverage) for Alloy models in the spirit of traditional unit testing. This paper describes our Java implementation of ASketch, which is a command-line tool, released as an open-source project on GitHub. Our experimental results show that ASketch can handle partial Alloy models with multiple holes and a large search space. The demo video for ASketch can be found at https://youtu.be/T5NIVsV329E.

show abstract