Associating architectural issues with quality attributes

Anvaari, Mohsen; Sørensen, Carl-Fredrik; Zimmermann, Olaf

doi:10.1145/2993412.3004847

Cited by 2 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, studying tool evaluations in a single company or within a single domain could provide a limited view on the criteria. Anvaari et al [1] reported that neither long experience in the area of interest nor the same domain of expertise provided agreement among raters.…”

Section: Discussionmentioning

confidence: 99%

“…In software engineering (SE), there are studies reporting low values for expert agreement/reliability using Kirppendorff's alpha and/or ICC, by e.g., Borg et al [4], Anvaari et al [1] and Kitchenham et al [27]. Evaluations depend on the interpretation of a construct under study, i.e., include some degree of subjectivity [5,47].…”

Section: Assessment Of Responsesmentioning

confidence: 99%

“…Krippendorff's alpha (α) is a statistical measure for determining inter-rater reliability. The values for the α range from perfect disagreement (0) to perfect agreement (1). The values α ≥ 0.800 are suggested for drawing reliable conclusions while values 0.667 ≤ α < 0.800 are claimed for tentative conclusions only [29].…”

Section: Krippendorff's Alphamentioning

confidence: 99%

“…As our data was considered to be of type ratio, but was limited to our scale (values from 0 to 10 at intervals of 0.5, i.e., 21 levels), we calculated the CV for both ratio and ordinal type of data. For ratio type of data the CV was calculated as the ratio of the standard deviation to the mean (1). For calculating the CV for the ordinal type of data we used the formula (2) presented by Kvålseth [31].…”

Section: Coefficient Of Variation (Cv)mentioning

confidence: 99%

See 3 more Smart Citations

Practitioner Evaluations on Software Testing Tools

Raulamo-Jurvanen

Hosio

Mäntylä

2019

Proceedings of the Evaluation and Assessment on Software Engineering

View full text Add to dashboard Cite

In software engineering practice, evaluating and selecting the software testing tools that best fit the project at hand is an important and challenging task. In scientific studies of software engineering, practitioner evaluations and beliefs have recently gained interest, and some studies suggest that practitioners find beliefs of peers more credible than empirical evidence. To study how software practitioners evaluate testing tools, we applied online opinion surveys (n=89). We analyzed the reliability of the opinions utilizing Krippendorff's alpha, intra-class correlation coefficient (ICC), and coefficients of variation (CV). Negative binomial regression was used to evaluate the effect of demographics. We find that opinions towards a specific tool can be conflicting. We show how increasing the number of respondents improves the reliability of the estimates measured with ICC. Our results indicate that on average, opinions from seven experts provide a moderate level of reliability. From demographics, we find that technical seniority leads to more negative evaluations. To improve the understanding, robustness, and impact of the findings, we need to conduct further studies by utilizing diverse sources and complementary methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Assessment Of Responsesmentioning

confidence: 99%

Section: Krippendorff's Alphamentioning

confidence: 99%

Section: Coefficient Of Variation (Cv)mentioning

confidence: 99%

See 2 more Smart Citations

Practitioner Evaluations on Software Testing Tools

Raulamo-Jurvanen

Hosio

Mäntylä

2019

Proceedings of the Evaluation and Assessment on Software Engineering

View full text Add to dashboard Cite

show abstract

“…Scenarios used in software architecture evaluation have been classified into various categories, such as direct scenarios, indirect scenarios, complex scenarios, use case scenarios, growth scenarios, and exploratory scenarios [9], [27], [35], [37]. The Software Engineering Institute (SEI) has enumerated a collection of general quality attribute scenarios intended to help characterize most commonly known quality attributes [6], [10]. A general scenario is, in effect, a template for generating a specific quality attribute scenario.…”

Section: A Software Architecture Evaluationmentioning

confidence: 99%

An Empirical Study of the Effectiveness of Software Architecture Evaluation Meetings

Babar¹,

Shen²,

Biffl³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

One of the most important issues in scenario-based software architecture evaluation is the development of scenarios for characterizing a desired set of quality attributes by holding meetings of stakeholders. As such meetings are costly to arrange, studying the effectiveness of such meetings is an important research question. In this paper, we report on the findings from a software architecture evaluation empirical study aimed at investigating the effectiveness of scenario development meetings in terms of the quantity of gained and lost scenarios during the evaluation process. The findings from the experiment data analysis raise the question about the effectiveness of holding meetings for developing scenarios as more existing scenarios were lost than new scenarios were gained as a result of these meetings. INDEX TERMS Architecture evaluation, process improvement, controlled experiment, scenario development, empirical study.

show abstract

Associating architectural issues with quality attributes

Cited by 2 publications

References 25 publications

Practitioner Evaluations on Software Testing Tools

Practitioner Evaluations on Software Testing Tools

An Empirical Study of the Effectiveness of Software Architecture Evaluation Meetings

Contact Info

Product

Resources

About