Experimentation in Software Engineering

Basili, Victor R.; Selby, Richard W.; Hutchins, David H.

doi:10.1007/3-540-27662-9_19

Cited by 26 publications

(37 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, our research program will also attempt to understand the socio-technical aspects of using groupware systems in the software architecture evaluation process. The design of this research project is based on a framework of experimentation (Basili et al 1986) with guidelines provided in (Kitchenham et al 2002).…”

Section: Introductionmentioning

confidence: 99%

Comparing distributed and face-to-face meetings for software architecture evaluation: A controlled experiment

2007

View full text Add to dashboard Cite

Scenario-based methods for evaluating software architecture require a large number of stakeholders to be collocated for evaluation meetings. Collocating stakeholders is often an expensive exercise. To reduce expense, we have proposed a framework for supporting software architecture evaluation process using groupware systems. This paper presents a controlled experiment that we conducted to assess the effectiveness of one of the key activities, developing scenario profiles, of the proposed groupware-supported process of evaluating software architecture. We used a cross-over experiment involving 32 teams of three 3rd and 4th year undergraduate students. We found that the quality of scenario profiles developed by distributed teams using a groupware tool were significantly better than the quality of scenario profiles developed by face-to-face teams (p<0.001). However, questionnaires indicated that most participants preferred the face-to-face arrangement (82%) and 60% thought the distributed meetings were less efficient. We conclude that distributed meetings for developing scenario profiles are extremely effective but that tool support must be of a high standard or participants will not find distributed meetings acceptable.

show abstract

Section: Introductionmentioning

confidence: 99%

Comparing distributed and face-to-face meetings for software architecture evaluation: A controlled experiment

2007

View full text Add to dashboard Cite

show abstract

“…The organization of each group of subjects 3 in each exper- 3 The students were assigned to the four groups in a randomly balanced way. imental lab session (Lab1 and Lab2) followed the design shown in Table 4.…”

Section: Variable Selection and Experiments Designmentioning

confidence: 99%

“…In the context of this experiment, we considered different tasks and different subjects as compared to the experiments on comprehension activities. According to the two-dimensional classification scheme by Basili et al [3], we performed blocked subject-project studies, as we examined objects across a set of subjects and a set of projects. The description follows a template originating from the Goal-Question-Metric (GQM) paradigm [2] as described by Wohlin et al [26].…”

Section: Design Of the Experimentsmentioning

confidence: 99%

A fine-grained analysis of the support provided by UML class diagrams and ER diagrams during data model maintenance

et al. 2013

View full text Add to dashboard Cite

This paper presents the results of an empirical study aiming at comparing the support provided by ER and UML class diagrams during maintenance of data models. We performed one controlled experiment and two replications that focused on comprehension activities (the first activity in the maintenance process) and another controlled experiment on modification activities related to the implementation of given change requests. The results achieved were analyzed at a fine-grained level aiming at comparing the support given by each single building block of the two notations. Such an analysis is used to identify weaknesses (i.e., building blocks not easy to comprehend) in a notation and/or can justify the need of preferring ER or UML for data modeling. The analysis revealed that the UML class diagrams generally provided a better support for both comprehension and modification activities performed on data models as compared to ER diagrams. Nevertheless, the former has some weaknesses related to three building blocks, i.e., multi-value attribute, composite attribute, and weak entity. These findings suggest that an extension of UML class diagrams should be considered to Communicated by Prof. Tony Clark and Prof. Jon Whittle. This paper is an extension of the work "Identifying the Weaknesses of UML Class Diagrams during Data Model Comprehension" appeared in the

show abstract

“…Following the template set out by the goal-question-metric (GQM) paradigm (Basili et al 1986), the research objective of this study is to: "Analyse ad-hoc and systematic V&V technologies, as well as automated static analysis tools applied by novices for the purpose of comparison with respect to their cost-effectiveness from the point of view of the researcher and practitioner in the context of concurrent Java components." There are two dimensions (or factors) of interest in the experiment: the use of automated static analysis tools and the application of TestCon steps (as depicted in Table 1).…”

Section: Goals Hypotheses and Metricsmentioning

confidence: 99%

Maximising the information gained from a study of static analysis technologies for concurrent software

Wojcicki

Strooper

2007

Empir Software Eng

View full text Add to dashboard Cite

The results of empirical studies in Software Engineering are limited to particular contexts, difficult to generalise and the studies themselves are expensive to perform. Despite these problems, empirical studies can be made effective and they are important to both researchers and practitioners. The key to their effectiveness lies in the maximisation of the information that can be gained by examining and replicating existing studies and using power analyses for an accurate minimum sample size. This approach was applied in a controlled experiment examining the combination of automated static analysis tools and code inspection in the context of the verification and validation (V&V) of concurrent Java components. The paper presents the results of this controlled experiment and shows that the combination of automated static analysis and code inspection is cost-effective. Throughout the experiment a strategy to maximise the information gained from the experiment was used. As a result, despite the size of the study, conclusive results were obtained, contributing to the research on V&V technology evaluation.

show abstract

Experimentation in Software Engineering

Cited by 26 publications

References 55 publications

Comparing distributed and face-to-face meetings for software architecture evaluation: A controlled experiment

Comparing distributed and face-to-face meetings for software architecture evaluation: A controlled experiment

A fine-grained analysis of the support provided by UML class diagrams and ER diagrams during data model maintenance

Maximising the information gained from a study of static analysis technologies for concurrent software

Contact Info

Product

Resources

About