Mining precise performance-aware behavioral models from existing instrumentation

Abstract

Software bugs often arise from differences between what developers envision their system does and what that system actually does. When faced with such conceptual inconsistencies, debugging can be very difficult. Inferring and presenting developers with accurate behavioral models of the system implementation can help developers reconcile their view of the system with reality and improve system quality.We present Perfume, a model-inference algorithm that improves on the state of the art by using performance information to differentiate otherwise similar-appearing executions and to remove false positives from the inferred models. Perfume uses a system's runtime execution logs to infer a concise, precise, and predictive finite state machine model that describes both observed executions and executions that have not been observed but that the system can likely generate. Perfume guides the model inference process by mining temporal performance-constrained properties from the logs, ensuring precision of the model's predictions. We describe the model inference process and demonstrate how it improves precision over the state of the art.

Discovering specification violations in networked software systems

“…APE uses Synoptic [9] to convert observed execution traces into a finite state machine (FSM) model of the target's behavior. (While APE could instead use other model-inference algorithms [6], [7], [8], [20], [23], [25], [32], [37], [39], [40], [41], [42], [43], [48], our experience showed that Synoptic model's enforcement of observed temporal invariants leads to sufficiently precise models for APE's purposes.) Each path through the FSM model represents an execution, in terms of the sequence of messages sent and received by APE.…”

“…There are many existing techniques for such model inference [6], [8], [9], [20], [23], [25], [37], [39], [41], [42], [43], [48], and it is not the focus of this paper to improve on them. Instead, APE uses Synoptic because of its precise predictive properties and previous use for manual software debugging [9].…”

Visually reasoning about system and resource behavior

“…Perfume's interactive interface builds on the Perfume algorithm and prototype [16,17]. Other model-inference algorithms, e.g., [3,4,5,6,14,15], can benefit from similar interfaces.…”

Inferring models of concurrent systems from logs of their behavior with CSight

“…For logs of serial systems (totally ordered logs), the problem of automata inference from positive examples of executions is computable [12] but NP-complete [30,5], and the FSA cannot be approximated by any polynomial-time algorithm [48]. Unlike CSight, prior work on model inference from totally ordered logs either excluded concurrency or captured a particular interleaving of concurrent events [2,10,7,11,18,28,43,46,49]. Tomte [1] and Synoptic [10] take a similar approach to CSight, using CEGAR [16] to refine models.…”