The rise and fall of the Chaos report figures

Abstract

I E E E S o f t w a r E P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y0 7 4 0 -7 4 5 9 / 1 0 / $ 2 6 . 0 0 © 2 0 1 0 I E E E focus management hardly help increase project success. Over the years, their figures have attracted tremendous attention.However, we question the validity of their figures. Robert Glass 2,3 and Magne Jørgensen and his colleagues 4 indicated that the only way to assess the Chaos results' credibility is to use Standish's data and reiterate their analyses. But there's another way: obtain your own data and reproduce Standish's research to assess its validity. We applied the Standish definitions to our extensive data consisting of 5,457 forecasts of 1,211 real-world projects totaling hundreds of millions of euros. Our research shows that the Standish definitions of successful and challenged projects have four major problems: they're misleading, one-sided, pervert the estimation practice, and result in meaningless figures. Misleading DefinitionsThe Standish Group published the first Chaos report in 1994, which summarized Standish's research findings and aimed to investigate causes of software project failure and find key ways to reduce such failures. 1 The group also intended to identify the scope of software project failures by defining three project categories that we recall verbatim:■ Resolution Type 1, or project success. The project is completed on time and on budget, offering all features and functions as initially specified. ■ Resolution Type 2, or project challenged. The project is completed and operational but over budget and over the time estimate, and offers fewer features and functions than originally specified. ■ Resolution Type 3, or project impaired. The project is cancelled at some point during the development cycle. 1To find answers to their research questions, Standish sent out questionnaires. Their total sample size was 365 respondents representing 8,380 applications. On the basis of the responses, Standish published overall percentages for each project cat-F or many years, researchers and practitioners have analyzed how to successfully manage IT projects. Among them is the Standish Group, which regularly publishes its findings in its Chaos reports. In 1994, Standish reported a shocking 16 percent project success rate, another 53 percent of the projects were challenged, and 31 percent failed outright.

Soft Skills for Software Project Team Members

“…It is noted that up to 70% of SDPs result in failure [16], [17]. The study conducted by Elmam and Koru [3] indicates that the top 5 causes of project failure are non-technical issues.…”

Strategic Business-IT alignment of application software packages: Bridging the Information Technology gap

“…This is in contrast to the 2004 study conducted by The Standish Group, which reported a higher success rate for IT projects at 29 per cent [18]. The Standish Chaos Report for 2006 showed that 35 per cent of IT projects were successful, which decreased by 3 per cent to a 32 per cent success rate, according to their 2009 study [18].…”

“…For example in a study conducted by PriceWaterhouseCoopers in 2004, 10 640 IT projects were surveyed and revealed that only 2.5 per cent of organisations achieve budget, scope and schedule targets in all IT projects [13]. This is in contrast to the 2004 study conducted by The Standish Group, which reported a higher success rate for IT projects at 29 per cent [18]. The Standish Chaos Report for 2006 showed that 35 per cent of IT projects were successful, which decreased by 3 per cent to a 32 per cent success rate, according to their 2009 study [18].…”

Identificación de factores que permitan potencializar el éxito de proyectos de desarrollo de software

“…El estudio del éxito de los proyectos de software ha sido un tema de gran atención desde hace más de una década, en la que se ha tratado de analizar los factores que colaboran al cumplimiento completo de un desarrollo y la manera como las empresas tratan de evitar posibles fracasos. Un ejemplo fue el análisis realizado [1] sobre las estadísticas del Standish Group de proyectos exitosos donde evidenciaba que el crecimiento de efectividad de los proyectos entre 2000 y 2004 había sido solo de un 1%, situación que mejora al 2009 con un 4%. Cabe aclarar que con el paso de los años los factores tenidos en cuenta para el estudio han estado ampliándose para hallar con más efectividad una respuesta [2].…”