Pursuing coherence in software process assessment and improvement

Abstract

The improvement of software development processes is certainly one of the top priorities of the IT industry. Software has become the most critical component in any modern product and service. All sectors of our society have been affected and influenced by the introduction of software and software‐based features. To support the improvement of software development processes, a number of process assessment and improvement techniques have been proposed and applied in a significant number of cases. However, the evaluation of the effects produced by the introduction of assessment and improvement methods and models is raising much controversy among researchers and practitioners. In particular, it is extremely difficult to evaluate if, under what circumstances, and to what extent an assessment approach (and the consequent improvement plan) has had a direct, significant and positive impact on the company performance. This paper reports our experiences in conducting process assessment/improvement initiatives in three companies. The assessments have been based on SEI's Capability Maturity Model for software (Software CMM). The main finding of this study is that effective process assessment (and consequently improvement) models/methods must necessarily pursue the coherent improvement of all the aspects of a company and not just of the engineering factors. In the paper we provide a detailed discussion of the case studies and of the motivations that justify our position. Also, as a synthesis of our experiences, we outline a strategy to software process assessment and improvement that takes into account the concerns and requirements we have identified in our work. Copyright © 2001 John Wiley & Sons Ltd

A Taxonomy to Compare SPI Frameworks

“…Bamford and Deibler [3], and Paulk [21] both compare ISO 9000 with CMM. Cattaneo, Fuggetta and Sciuto [6] compare CMM, MBA and ISO 9000. Ares et al propose the ATAM method to compare assessment frameworks according to six main categories [1].…”

“…We present a list of 25 characteristics regarded as important for describing SPI frameworks. Sørumgård [24] and Cattaneo et al [6] have been particularly influential on our work, but we have also tried to capture aspects generally regarded as important in SPI literature (as cited previously).…”

“…A guideline like this is often called a process improvement life cycle. It "defines the strategy and the process by which improvement is pursued" [6]. This characteristic indicates the type of improvement method included in the framework, if any.…”

“…Other approaches are easier for smaller companies to employ. However, this relation between organization size and costeffectiveness is hard to pinpoint and has been heavily debated [6] [8].…”

“…Coherence is the logical connection between engineering factors and factors related to the business or organization. In an article by Cattaneo et al [6] this was pointed out as extremely important for a successful SPI implementation, yet still insufficient in many process improvement frameworks. Cattaneo et al stated that "most software process improvement efforts limit their focus and concern to the technical and engineering aspects of software development [...] They do not consider the relationships between these engineering factors and organizational and market variables.…”