A resilience‐based framework for assessing the evolution of open source software projects

Abstract

Open source software (OSS) has been developing for more than two decades. It originated as a movement with the introduction of the first free/libre OSS operating system, became a popular trend among the developer community, led to enterprise solutions widely embraced by the global market, and began garnering attention from significant players in the software industry (such as IBM's acquisition of RedHat). Throughout the years, numerous software assessment models have been suggested, some of which were created specifically for OSS projects. Most of these assessment models focus on software quality and maintainability. Some models are taking under consideration health aspects of OSS projects. Despite the multitude of these models, there is yet to be a universally accepted model for assessing OSS projects. In this work, we aim to adapt the City Resilience Framework (CRF) for use in OSS projects to establish a strong theoretical foundation for OSS evaluation focusing on the project's resilience as it evolves over time. We would like to highlight that our goal with the proposed assessment model is not to compare two OSS solutions with each other, in terms of resilience, or even do a resilience ranking between the available OSS tools. We are aiming to investigate resilience of an OSS project as it evolves and identify possible opportunities of improvements in the four dimensions we are defining. These dimensions are as follows: source code, business and legal, integration and reuse, and social (community). The CRF is a framework that was introduced to measure urban resilience and most specifically how cities' resilience is changing as they evolve. We believe that a software evaluation model that focuses on resilience can complement the pre‐existing models based on software quality and software health. Although concepts that are related to resilience, like sustainability or viability, already appear in literature, to our best knowledge, there is no OSS assessment model that evaluates the resilience of an OSS project. We argue that cities and OSS projects are both dynamically evolving systems with similar characteristics. The proposed framework utilizes both quantitative and qualitative indicators, which is viewed as an advantage. Lastly, we would like to emphasize that the framework has been tested on the enterprise software domain as part of this study, evaluating five major versions of six OSS projects, Laravel, Composer, PHPMyAdmin, OKApi, PatternalPHP, and PHPExcel, the first three of which are intuitively considered resilient and the three latter nonresilient, to provide a preliminary validation of the models' ability to distinguish between resilient and not resilient projects.