Assessing Software Development Teams' Efficiency using Process Mining

Abstract

Context. Improving the efficiency and effectiveness of software development projects implies understanding their actual process. Given the same requirements specification, different software development teams may follow different strategies and that may lead to inappropriate use of tools or non-optimized allocation of effort on spurious activities, non-aligned with the desired goals. However, due to its intangibility, the actual process followed by each developer or team is often a black box.Objective. The overall goal of this study is to improve the knowledge on how to measure efficiency in development teams where a great deal of variability may exist due to the humanfactor. The main focus is on the discovery of the underlying processes and compare them in terms of efficiency and effectiveness. By doing so, we expect to reveal potentially hidden costs and risks, so that corrective actions may take place on a timely manner during the software project life cycle.Method. Several independent teams of Java programmers, using the Eclipse IDE, were assigned the same software quality task, related to code smells detection for identifying refactoring opportunities and the quality of the outcomes were assessed by independent experts. The events corresponding to the activity of each team upon the IDE, while performing the given task, were captured. Then, we used process mining techniques to discover development process models, evaluate their quality and compare variants against a reference model used as "best practice".Results. Teams whose process model was less complex, had the best outcomes and vice-versa. Comparing less complex process variants with the "best practice" process, showed that they were also the ones with less differences in the control-flow perspective, based on activities frequencies. We have also determined which teams were most efficient through process analysis.Conclusions. We confirmed that, even for a well-defined software development task, there may be a great deal of process variability due to the human factor. We were able to identify when developers were more or less focused in the essential tasks they were required to perform. Less focused teams had the more complex process models, due to the spurious / non-essential actions that were carried out. In other words, they were less efficient. Experts' opinion confirmed that those teams also were less effective in their expected delivery. We therefore concluded that a self-awareness of the performed process rendered by our approach, may be used to identify corrective actions that will improve process efficiency (less wasted effort) and may yield to better deliverables, i.e. improved process effectiveness.

Process Mining Discovery Techniques for Software Architecture Lightweight Evaluation Framework

“…The aim is to retrieve control-flow and structural aspects of software development cycles from existing data sources and software repositories. As a result, Bug trackers, version control systems, and mail archives are related to tracking software development events [6][7][8][9][10][11][12][13].…”

Angewandte Forschung in der Wirtschaftsinformatik 2022

“…[Ru07] identified the possibility of a 'faster PMS creation'. [Ru14b] suggested to embed the model into the development lifecycle due to effectiveness, [Ca19] found that their model accuracy was sufficient. The other works did at least not falsify this assumption and presented theoretical or practical benefits e.g.…”

Profiling Software Developers with Process Mining and N-Gram Language Models

“…A development session executed by one developer at his/her IDE can be considered an instance of a process, where the goal is to produce a software product or maintain an existing one. Its workflow of activities depends on many factors, such as the development methodology, program design or individual experience [10]. Furthermore, developers are usually free to produce code without a referential model or guidelines on how to execute the coding tasks and, most often without any intelligent guidance from traditional development tools.…”