The scenario coevolution paradigm: adaptive quality assurance for adaptive systems

Abstract

Systems are becoming increasingly more adaptive, using techniques like machine learning to enhance their behavior on their own rather than only through human developers programming them. We analyze the impact the advent of these new techniques has on the discipline of rigorous software engineering, especially on the issue of quality assurance. To this end, we provide a general description of the processes related to machine learning and embed them into a formal framework for the analysis of adaptivity, recognizing that to test an adaptive system a new approach to adaptive testing is necessary. We introduce scenario coevolution as a design pattern describing how system and test can work as antagonists in the process of software evolution. While the general pattern applies to large-scale processes (including human developers further augmenting the system), we show all techniques on a smaller-scale example of an agent navigating a simple smart factory. We point out new aspects in software engineering for adaptive systems that may be tackled naturally using scenario coevolution. This work is a substantially extended take on Gabor et al. (International symposium on leveraging applications of formal methods, Springer, pp 137-154, 2018).

Rigorous engineering of collective adaptive systems: special section

“…In "Adapting quality assurance to adaptive systems: the scenario coevolution paradigm" [35] by Thomas Gabor, Andreas Sedlmeier, Thomy Phan, Fabian Ritz, Marie Kiermeier, Lenz Belzner, Bernhard Kempter, Cornel Klein, Horst Sauer, Reiner Schmid, Jan Wieghardt, Marc Zeller, and Claudia Linnhoff-Popien take a more abstract view and present a formal framework for adaptation and testing of adaptive systems using scenarios; it also discusses how such a framework can be used for increasing the trustworthiness of complex adaptive systems. In particular, the work extends the system model and the notion of adaptation space of [41] of the ASCENS project [74] by abstract definitions of selfadaptation and scenarios.…”

On the Modeling and Verification of Collective and Cooperative Systems

“…In the literature, it is worth mentioning that formal, process-algebraic approaches ( Loreti and Hillston, 2016 ), semi-formal, architectural description approaches ( Ozkaya and Kloukinas, 2013 ), and combinations of both ( Basu et al, 2011 ; Hennicker et al, 2014 ; Bures et al, 2016 ) have been proposed to model and analyze dynamic reconfigurable architectures ( De Nicola et al, 2020 ) and (self-)adaptive systems ( Gabor et al, 2020 ). In particular, the language CARMA ( Loreti and Hillston, 2016 ) is specifically defined to model collective adaptive systems and shares several features with our framework, such as the separation of concerns advocated in Section 2 , support for local/global views, and a formal semantics in operational style.…”

Ensemble-Based Modeling Abstractions for Modern Self-optimizing Systems

“…In [17], a theorem defining a theoretical bound on the impact of applying a machinelearning method during adaptation was defined, and an approach for reducing an adaptation space was proposed. The paper [15] proposes a framework for the coevolution of an adaptive system together with its tests. Machine-learning is used for restrictions of an adaptation space in order to achieve a meaningful system as a result of an adaption step.…”