A Literature Study to Characterize Continuous Experimentation in Software Engineering

Abstract

Continuous Experimentation (CE) has become increasingly popular across industry and academic communities. Given its rapid evolution in software engineering (SE), the lack of a common understanding of CE can jeopardize new implementations and justify research efforts. Therefore, this literature study characterizes CE in SE based on its definitions, processes, and strategies for experimentation available in the technical literature. Seventy-six sources of information provided many different definitions, processes, and experimental procedures used to describe CE in SE. Despite the increasing use of CE in SE, it is impossible to observe a common terminology yet to support its characterization and use.

Ethical Issues in Continuous Experimentation

“…Lastly, they guide software development processes, ensuring continuous improvement and adaptation based on user data and feedback. In this context, learning about customers, users, and the market takes precedence over the code itself, emphasizing the importance of customer-centricity [Erthal et al 2023].…”

“…Furthermore, as online experimentation evolves into a continuous model, the impact of these ethical issues becomes even more significant. The continuous nature of online experimentation implies that participants are constantly exposed to interventions and changes in the digital environment [Erthal et al 2023]. Therefore, we need to understand the concepts and definitions that involve experimentation to advance the discussion toward continuous experimentation.…”