Classification and evaluation of defects in a project retrospective

Leszak, Marek; Perry, Dewayne E.; Stoll, Dieter

doi:10.1016/s0164-1212(01)00146-7

Cited by 112 publications

(88 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Defect studies on other large software systems have shown similar results as far as the occurrence of these defects in software [5], [20], [22].…”

Section: Type and Trigger/typementioning

confidence: 60%

“…The use of causal analysis to investigate subsets of defects belonging to patterns of interest is typical of both ODC [5], [10] and of defect analysis [20], [25]. Often the diagnosis is performed by experts gathered to investigate the reasons for the patterns and how such patterns of defects can be prevented or controlled in the future.…”

Section: Resultsmentioning

confidence: 99%

“…However, when Leszak et al performed a large, retrospective causalanalysis study of defects through the load-building phase (i.e., not during operations), they found that implementation defects consumed 75 percent of all effort. The defects were primarily of type algorithm or functionality [20]. Ostrand and Weyuker recently compared pre and postrelease faults to investigate module fault density and fault proneness in an inventory-tracking system [28].…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Empirical analysis of safety-critical anomalies during operations

Lutz

Mikulski

2004

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Abstract-Analysis of anomalies that occur during operations is an important means of improving the quality of current and future software. Although the benefits of anomaly analysis of operational software are widely recognized, there has been relatively little research on anomaly analysis of safety-critical systems. In particular, patterns of software anomaly data for operational, safety-critical systems are not well understood. This paper presents the results of a pilot study using Orthogonal Defect Classification (ODC) to analyze nearly two hundred such anomalies on seven spacecraft systems. These data show several unexpected classification patterns such as the causal role of difficulties accessing or delivering data, of hardware degradation, and of rare events. The anomalies often revealed latent software requirements that were essential for robust, correct operation of the system. The anomalies also caused changes to documentation and to operational procedures to prevent the same anomalous situations from recurring. Feedback from operational anomaly reports helped measure the accuracy of assumptions about operational profiles, identified unexpected dependencies among embedded software and their systems and environment, and indicated needed improvements to the software, the development process, and the operational procedures. The results indicate that, for long-lived, critical systems, analysis of the most severe anomalies can be a useful mechanism both for maintaining safer, deployed systems and for building safer, similar systems in the future.

show abstract

“…Defect studies on other large software systems have shown similar results as far as the occurrence of these defects in software [5], [20], [22].…”

Section: Type and Trigger/typementioning

confidence: 60%

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Empirical analysis of safety-critical anomalies during operations

Lutz

Mikulski

2004

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Weiss, et al have cited requirements misunderstanding in several recent disasters, stating, "software-related accidents almost always are due to misunderstanding about what the software should do" [15]. Similarly, in summarizing the results of a large defect-analysis study, Leszak, et al stated that "domain and system knowledge continue to be one of the largest underlying problems in software development" [7]. Curtis, et al have also implicated the "...thin spread of application domain knowledge" as a main limiting factor to software productivity and quality in the design of large systems [1].…”

Section: Related Workmentioning

confidence: 99%

Using occurrence properties of defect report data to improve requirements

Wasson

Schmid

Lutz

et al. 2005

13th IEEE International Conference on Requirements Engineering (RE'05)

View full text Add to dashboard Cite

Defect reports generated for faults found during testing provide a rich source of information regarding problematic phrases used in requirements documents. These reports indicate that faults often derive from instances of ambiguous, incorrect or otherwise deficient language. In this paper, we report on a method combining elements of linguistic theory and information retrieval to guide the discovery of problematic phrases throughout a requirements specification, using defect reports and correction requests generated during testing to seed our detection process. We found that phrases known from these materials to be problematic have occurrence properties in requirements documents that both allow the direction of resources to prioritize their correction, and generate insights characterizing more general locations of difficulty within the requirements. Our findings lead to some recommendations for more efficiently and effectively managing certain natural language issues in the creation and maintenance of requirements specifications.

show abstract

“…At least for largescale and complex systems this is often not adequate. If the data would be collected and analyzed on subsystem level (product) or team level (process/project) some interesting insights can be revealed [1,2], not available if the system is studied from a 'black-box' perspective. Characteristics of different subsystems tend often not to be homogenous, causally due to e.g.…”

mentioning

confidence: 99%

Extending Empirical Studies to Cover More Realistic Industrial Development and Project Management Issues

Leszak¹

Empirical Software Engineering Issues. Critical Assessment and Future Directions

Self Cite

View full text Add to dashboard Cite

"Real development" issues, observable in many medium size to large size software systems need to be tackled more intensively by empirical research, to increase applicability and concrete transfer of scientific results into practice. This is illustrated by three areas, covering typical problems in today's software project management.Many predictive and prescriptive models rely on the assumption of steady-state project dynamics, at least for main driving factors. But industrial software development is facing an increasing amount of global development, offshore outsourcing, high turnover rate, requirements volatility, constrained resources (both in interval and in cost), mental project pressure, unforeseen project events, etc..Granularity of empirical studies is often on overall system level. At least for largescale and complex systems this is often not adequate. If the data would be collected and analyzed on subsystem level (product) or team level (process/project) some interesting insights can be revealed [1,2], not available if the system is studied from a 'black-box' perspective. Characteristics of different subsystems tend often not to be homogenous, causally due to e.g. different team size and expertise, and architectural dependencies. E.g. one of our case studies on defect and file distribution [2] provided evidence that a) per subsystem, delivered defects are a good estimator of in-process defects, and 2) per subsystem, the number of product requirements is highly correlated with in-process defects. In this case study such significance could neither be observed on system-level nor on module-level.Another often neglected aspect of current empirical research is their scope w.r.t. the lifecycle phases studied. If extended towards overall development lifecycle the research results would widen their scope, leading to intensified and value-added collaboration with industrial software development. E.g. most research results on software reviews and inspections focus on code artifacts, although in real development projects typically a much higher amount of effort is spent on reviewing requirements, design documents, and test plans. It is not surprising to see evidence for review effectivity & efficiency being highly dependent (also) on type of artifact [3].These exemplified levels of detail, in turn, requires on the industrial organization a high degree of data quality, typically not found in lower maturity organizations. Including more realistic factors from large software systems and real-life projects into empirical studies would improve model adequacy and their predictive power.

show abstract

Classification and evaluation of defects in a project retrospective

Cited by 112 publications

References 10 publications

Empirical analysis of safety-critical anomalies during operations

Empirical analysis of safety-critical anomalies during operations

Using occurrence properties of defect report data to improve requirements

Extending Empirical Studies to Cover More Realistic Industrial Development and Project Management Issues

Contact Info

Product

Resources

About