Testing and inspecting reusable product line components

Abstract

In recent years, product line development has increasingly received attention in industry as it enables software-developing organizations to reduce both cost and time of developing and maintaining increasingly complex systems as well as to address the demands for individually customized products. Successful product line development requires high quality of reusable artifacts in order to achieve the promised benefits. The unique issues of quality assurance in the context of systematic reuse, however, have not been quantitatively investigated so far. This paper describes a first empirical study comparing the two defect detection techniques, code inspections and functional testing, in the context of product line development. The primary goal of the study was to initially investigate the defect finding potential of the techniques on reusable software components with common and variant features. The major findings of the study are that the two techniques identified different types of defects on variants of a reusable component. Inspections are on average 66.39% more effective and need on average 36.84% less effort to detect a defect We found that both the testing and inspection techniques applied in the experiment were ineffective in identifying variant-specific defects. Overall, the results indicate that the standard quality assurance techniques seem to be insufficient to address special characteristics of reusable components.

Software product line testing – A systematic mapping study

“…Open items Hook and template coverage criteria for testing frameworkbased software product families Conceptual proposal Metric Denger and Kolb [11] Testing and inspecting reusable product line components: first empirical results…”

Experiences with Software Product Line Development in Risk Management Software

“…In fact, changes to core assets can lead to the retesting of all implementations of that asset which can be a drain on the resources of the quality assurance team. This appears to be an unsolved problem in the SPL community [26]. Applying the concepts of SPL development to the testing process as we have already applied it to the specification process may result in improvements in this area.…”

Improving the Testing and Testability of Software Product Lines

“…Other work on software testing product lines includes that of Denger et al [10] who present an empirical study to evaluate the difficulty of detecting faults in the common versus variable portions of an SPL code base concluding that the types of faults found in these two portions of the code differ. They use both white and black box techniques but do not test from the feature model.…”

“…A large body of research on SPL engineering has focused on reuse of core program assets [4,15,17], refined feature modeling [8,9,23], and code generation techniques [2,7]. There has also been research on testing software product lines [3,6,10,24].…”