Architectural self-organisation, in which different configurations of software modules are dynamically assembled based on the current context, has been shown to be an effective way for software to self-optimise over time. Current approaches to this rely heavily on human-led definitions: models, policies and processes to control how self-organisation works. We present the case for a paradigm shift to fully emergent computer software which places the burden of understanding entirely into the hands of software itself. These systems are autonomously assembled at runtime from discovered constituent parts and their internal health and external deployment environment continually monitored. An online, unsupervised learning system then uses runtime adaptation to continuously explore alternative system assemblies and locate optimal solutions. Based on our experience over the last three years, we define the problem space of emergent software and present a working case study of an emergent web server as a concrete example of the paradigm. Our results demonstrate two main aspects of the problem space for this case study: that different assemblies of behaviour are optimal in different deployment environment conditions; and that these assemblies can be autonomously learned from generalised perception data while the system is online.

Abstract-Architectural self-organisation, in which different configurations of software modules are dynamically assembled based on the current context, has been shown to be an effective way for software to self-optimise over time. Current approaches to this rely heavily on human-led definitions: models, policies and processes to control how self-organisation works. We present the case for a paradigm shift to fully emergent computer software which places the burden of understanding entirely into the hands of software itself. These systems are autonomously assembled at runtime from discovered constituent parts and their internal health and external deployment environment continually monitored. An online, unsupervised learning system then uses runtime adaptation to explore alternative system assemblies and locate optimal solutions. Based on our experience to date, we define the problem space of emergent software, and we present a working case study of an emergent web server. Our results demonstrate two aspects of the problem space for this case study: that different assemblies of behaviour are optimal in different deployment environment conditions; and that these assemblies can be autonomously learned from generalised perception data while the system is online.

Major research venues on autonomic and selfadaptive systems have been active for 16 years, exploring and building on the seminal vision of autonomic computing in 2003. We study the current trajectory and progress of the research field towards this vision, surveying on the research questions that are asked by researchers and the methodological practice that they employ in order to answer these questions. We survey contributions under this lens across the three main venues for primary research in autonomic and self-adaptive systems work: ICAC, SASO, and SEAMS. We examine the last three years of contributions from each venue, totaling 210 publications, to gain an understanding of the dominant current research questions and methodological practice-and what this shows us about the progress of the field. Our major findings include the general research questions focusing one level below the highest autonomy vision; methodological practice being split almost evenly between real-world experiments and simulation; a general a high positive results bias in publications; and a potential concern over low levels of repeatability across most contributions.

Contemporary systems are increasingly complex, with both large codebases and constantly changing environments which make them challenging to develop, deploy and manage. We consider two recent efforts to tackle this complexity: microservices and emergent software. Microservices have gained recent popularity in industry, in which monoliths of software are broken down into compositions of single-objective, end-to-end services running on HTTP which can be scaled out on cloud hosting systems. From the research community, the emergent systems concept demonstrates promise in using real-time learning to autonomously compose and optimise software systems from small building blocks, rapidly finding the best behavioural composition to match the current deployment conditions. We argue that emergent software and microservice architectures have strong potential for synergy in complex systems, offering mutually compatible lessons in dealing with complexity via scale-out design and real-time client-tailored behaviour. We explore self-designing microservices, built with emergent software, to demonstrate the complementary boundaries of both concepts-and how future intersections may offer novel architectures that lie at a compelling point between human-and machine-designed systems. We present the conceptual synergy and demonstrate a specific microservice architecture for a smart city example where scoped microservices are continually self-composed according to the demands of the applications and operating environment. For the purpose of reproducibility of the study, we make available all the code used in the evaluation of the proposed approach.

Emergent software systems take a reward signal, an environment signal, and a collection of possible behavioural compositions implementing the system logic in a variety of ways, to learn in real-time how best to assemble a system. This reduces the burden of complexity in systems building by making human programmers responsible only for developing potential building blocks while the system determines how best to use them in its deployment conditions-with no architectural models or training regimes. In this paper we generalise the approach to distributed systems, to demonstrate for the first time how a single reward signal can form the basis of complex decision making about how to compose the software running on each host machine, where to place each sub-unit of software, and how many instances of each sub-unit should be created. We provide an overview of the necessary system mechanics to support this concept, and discuss the key challenges in machine learning needed to realise it. We present our current implementation in both datacentre and pervasive computing environments, with experimental results for a baseline learning approach.