Identifying the implied: Findings from three differentiated replications on the use of security requirements templates

Abstract

Context: Identifying security requirements early on can lay the foundation for secure software development. Security requirements are often implied by existing functional requirements but are mostly left unspecified. The Security Discoverer process automatically identifies security implications of individual requirements sentences and suggests applicable security requirements templates.Goal: To support requirements analysts in identifying security requirements by automating the suggestion of security requirements templates that are implied by existing functional requirements. Method:We conducted a controlled experiment in a graduate-level security class at North Carolina State University (NCSU) to evaluate the Security Discoverer (SD) process in eliciting implied security requirements in 2014. We have subsequently conducted three differentiated replications to evaluate the generalizability and applicability of the initial findings. The replications were conducted across three countries at the University of Trento, NCSU, and the University of Costa Rica. We evaluated the responses of the 205 total participants in terms of quality, coverage, relevance and efficiency. We also develop shared insights regarding the impact of context factors such as time, motivation and support, on the study outcomes and provide lessons learned in conducting the replications.Results: Treatment group, using the SD process, performed significantly better than the control group (at p-value <0.05) in terms of the coverage of the identified security requirements and efficiency of the requirements elicitation process in two of the three replications, supporting the findings of the original study. Participants in the treatment group identified 84% more security requirements in the oracle as compared to the control group on average. Overall, 80% of the 111 participants in the treatment group were favorable towards the use of templates in identifying security requirements. Our qualitative findings indicate that participants may be able to differentiate between relevant and extraneous templates suggestions and be more inclined to fill in the templates with additional support. Conclusion:Security requirements templates capture the security knowledge of multiple experts and can support the security requirements elicitation process when automatically suggested, making the implied security requirements more evident. However, individual participants may still miss out on identifying a number of security requirements due to empirical constraints as well as potential limitations on knowledge and security expertise.

Human Aspect of Threat Analysis: A Replication

“…Several studies have empirically compared [17,18,23] and conducted replications [16,24,26] requirement engineering techniques (e.g., requirements elicitation). For brevity, we direct the interested reader a comprehensive review by Ambreen et al [1].…”

Secure MEReq: A Tool Support to Check for Completeness of Security Requirements

“…In Figure 4 completeness status for each requirement (7) (8). They can also view the examples of each component if needed (11).…”

Towards a Platform for Empirical Software Design Studies

“…Furthermore, attention should be given to the process of deploying updates with bug fixes. A web-based participant tool (e.g., [13], [14]) can be updated much more easily than a stand-alone tool (e.g., [10], [11]), for example.…”

“…Currently, the necessary tools are created often for the specific purpose of single study, e.g. [10]- [14], and these tools are labor-intensive to create, test, maintain, evolve and reuse.…”