Synthetic data generation for statistical testing

Nagy

Proceedings of the 40th International Conference on Software Engineering

2018

Many testing and benchmarking scenarios in software and systems engineering depend on the systematic generation of graph models. For instance, tool qualification necessitated by safety standards would require a large set of consistent (well-formed or malformed) instance models specific to a domain. However, automatically generating consistent graph models which comply with a metamodel and satisfy all well-formedness constraints of industrial domains is a significant challenge. Existing solutions which map graph models into first-order logic specification to use back-end logic solvers (like Alloy or Z3) have severe scalability issues. In the paper, we propose a graph solver framework for the automated generation of consistent domain-specific instance models which operates directly over graphs by combining advanced techniques such as refinement of partial models, shape analysis, incremental graph query evaluation, and rule-based design space exploration to provide a more efficient guidance. Our initial performance evaluation carried out in four domains demonstrates that our approach is able to generate models which are 1-2 orders of magnitude larger (with 500 to 6000 objects!) compared to mapping-based approaches natively using Alloy.

Section: Related Workmentioning

confidence: 99%

A graph solver for the automated generation of consistent domain-specific models

Nagy

Proceedings of the 40th International Conference on Software Engineering

2018

“…SDG [31] proposes an approach that uses a search-based custom OCL solver to generate synthetic data for statistical testing. Generated models are multidimensional and consistent.…”

Section: Related Workmentioning

confidence: 99%

“…Active research in automated graph model generation [10,25,30,31] has been focusing on deriving graphs with desirable properties like consistency, diversity, scalability or realistic nature [37]. A particularly challenging task of domainspecific model generators is to ensure consistency, i.e.…”

Section: Introductionmentioning

confidence: 99%

“…For example, SAT-solvers are used by Kodkod [34] that map high-level languages to propositional logic, CSP-solvers are exploited in EMF2CSP [10], while SMT-solvers were applied in [12,15,28]. Consistent model generators may rely on custom search-based techniques [31], symbolic techniques [25] or custom decision procedures [9,30] to improve scalability.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automated Generation of Consistent Graph Models with First-Order Logic Theorem Provers

Babikian

Fundamental Approaches to Software Engineering

2020

The automated generation of graph models has become an enabler in several testing scenarios, including the testing of modeling environments used in the design of critical systems, or the synthesis of test contexts for autonomous vehicles. Those approaches rely on the automated construction of consistent graph models, where each model satisfies complex structural properties of the target domain captured in first-order logic predicates. In this paper, we propose a transformation technique to map such graph generation tasks to a problem consisting of first-order logic formulae, which can be solved by state-of-the-art TPTPcompliant theorem provers, producing valid graph models as outputs. We conducted performance measurements over all 73 theorem provers available in the TPTP library, and compared our approach with other solver-based approaches like Alloy and VIATRA Solver.

“…There is a wide range of model generators such as Alloy [12], [13], Formula [14], [15], USE [16], UML2CSP [17], SDG [18], [19] and Viatra Solver [20], [21] to automatically derive consistent models for a given domain specification. Several generators are based on precise foundations offered by backend logic solvers (like SAT solvers [22], [23] or SMT solvers [24]).…”

Section: Introductionmentioning

confidence: 99%

Automated Generation of Consistent Graph Models With Multiplicity Reasoning

Marussy

IIEEE Trans. Software Eng.

2022

Advanced tools used in model-based systems engineering (MBSE) frequently represent their models as graphs. In order to test those tools, the automated generation of well-formed (or intentionally malformed) graph models is necessitated which is often carried out by solver-based model generation techniques. In many model generation scenarios, one needs more refined control over the generated unit tests to focus on the more relevant models. Type scopes allow to precisely define the required number of newly generated elements, thus one can avoid the generation of unrealistic and highly symmetric models having only a single type of elements. In this paper, we propose a 3-valued scoped partial modeling formalism, which innovatively extends partial graph models with predicate abstraction and counter abstraction. As a result, well-formedness constraints and multiplicity requirements can be evaluated in an approximated way on incomplete (unfinished) models by using advanced graph query engines with numerical solvers (e.g. IP or LP solvers). Based on the refinement of 3-valued scoped partial models, we propose an efficient model generation algorithm that generates models that are both well-formed and satisfy the scope requirements. We show that the proposed approach scales significantly better than existing SAT-solver techniques or the original graph solver without multiplicity reasoning. We illustrate our approach in a complex design-space exploration case study of collaborating satellites introduced by researchers at NASA Jet Propulsion Lab.