Sustainability of Systematic Literature Reviews

Abstract

Context: Systematic Literature Review (SLR) has been increasingly adopted in Software Engineering (SE) because it is considered the most reliable way to gather evidence from the literature. At the same time, SLR currently consumes much time and effort to be conducted, and its results have not impacted the software industry as expected. Despite several solutions proposed to solve the problems of SLR, the research remains very fragmented failing to provide a bigger picture of the problems and associated solutions. Objective: This study introduces the concept of sustainability in SLRs to the SE community as a holistic view of the current research scenario putting together a solid knowledge base that includes definitions, guidelines, and the main leveraging points of sustainability in current SLR conducted in SE. Method: After conducting a tertiary study investigating the main characteristics of sustainability of software systems, we applied the meta-ethnographic method to transpose sustainability characteristics into SLR context by identifying, adapting, synthesizing, and validating those which could aid researchers in conducting more useful studies with less effort. Finally, we extracted a sample of high-quality studies conducted in the SE area, next, we applied a sustainability view to identify the flaws that lead to unsustainable SLR. Results: Our study provides an overview of the role of sustainability in the SLR context and provides important assets for SE community to identify what are sustainable SLR, such as 15 characteristics of sustainable SLR and 15 sensitive points of SLR process that are considered critical factors to achieve sustainability. In addition, we propose 16 guidelines to support researchers to improve sustainability in their SLR and identified 13 leveraging points found in relevant studies in SE area that together represent actions that should be adopted immediately to enhance the sustainability of SLR. Conclusion: Our work represents a starting point for a mindset shift in the SE community about the importance of creating more sustainable reviews. Furthermore, we believe that sustainability is the missing key concept that could connect many advances achieved over the years of research and remove barriers that hinder SLR from providing useful information in a timely manner for academia and industry. Henceforth, we believe that researchers should observe sustainability as a first-class citizen while conducting/updating SLR to make them compatible with the sustainability view.