An Empirical Approach to Characterizing Risky Software Projects Based on Logistic Regression Analysis

Abstract

During software development, projects often experience risky situations. If projects fail to detect such risks, they may exhibit confused behavior. In this paper, we propose a new scheme for characterization of the level of confusion exhibited by projects based on an empirical questionnaire. First, we designed a questionnaire from five project viewpoints, requirements, estimates, planning, team organization, and project management activities. Each of these viewpoints was assessed using questions in which experience and knowledge of software risks are determined. Secondly, we classify projects into Bconfused'' and Bnot confused,'' using the resulting metrics data. We thirdly analyzed the relationship between responses to the questionnaire and the degree of confusion of the projects using logistic regression analysis and constructing a model to characterize confused projects. The experimental result used actual project data shows that 28 projects out of 32 were characterized correctly. As a result, we concluded that the characterization of confused projects was successful. Furthermore, we applied the constructed model to data from other projects in order to detect risky projects. The result of the application of this concept showed that 7 out of 8 projects were classified correctly. Therefore, we concluded that the proposed scheme is also applicable to the detection of risky projects.

Learning better IV&V practices

“…• Madachy's heuristic software risk model that alerts when certain observations are seen in a project [20]; • Boehm and Basili's top-10 defect reduction list [6,21] [29,30].…”

“…It is hard to make critical audit conclusions based on a Delphi-style analysis or inaccessible data. In general, only a minority of SE researchers can publish the data used to make their conclusions (for example, [22][23][24]29,30] Also, several of the these papers [27,28] only offer general advice about how to avoid problems in software development. The subjective nature of this advice makes it difficult to consistently deploy them over a national software development program.…”

“…The subjective nature of this advice makes it difficult to consistently deploy them over a national software development program. Only a few [23,24,29,30] offer publicdomain versions of their data and models.…”

Discriminating risky software project using neural networks

“…The former provides a quantifiable risk value so that project managers can discriminate between high, medium, or low risk software projects with respect to a given project risk level scale or by using clustering techniques [3][4][5]. The latter provides a meaningful sign (risky/not risky) as a classifier that a riskprone project can be effectively identified early and thus aid the planning of risk management strategies [6,7]. The latter approach was considered in this study.…”

“…Examples include logistic regression [6], Bayesian classification [8], and the association rule [9]. Although the overall classification accuracy of these approaches is at an acceptable level, correctly identifying a risky project at a true-positive rate is still a challenge.…”