Security and performance are critical nonfunctional requirements for software systems. Thus, it is crucial to include verification activities during software development to identify defects related to such requirements, avoiding their occurrence after release. Software verification, including testing and reviews, encompasses a set of activities that have a purpose of analyzing the software searching for defects. Security and performance verification are activities that look at defects related to these specific quality attributes. Few empirical studies have been focused on how is the state of the practice in security and performance verification. This paper presents the results of a case study performed in the context of Brazilian organizations aiming to characterize security and performance verification practices. Additionally, it provides a set of conjectures indicating recommendations to improve security and performance verification activities.

No abstract

Software testing aims to reveal failures due to the lack of conformity (defects) among functional and non-functional requirements and the implemented system. Thus, defects can be identified and fixed, improving software quality. However, despite several works emphasizing the importance of non-functional requirements (NFRs), there is an insufficient amount of software testing approaches dealing with them. The lack of NFR evaluation may be the cause of low-quality software that does not meet users need, influencing software project success. Goal: To organize a body of knowledge regarding NFRs and software testing approaches available in the technical literature and reveal the gaps between testable NFRs and software testing approaches. Method: To perform structured literature reviews to identify NFRs and software testing approaches dealing with testable NFRs. To combine both results, reveal research opportunities and organize a body of knowledge regarding NFRs and software testing approaches. Results: From 224 identified NFRs, 87 were described, and 47 software testing approaches observed. Only eight approaches are empirically evaluated. No testing approaches were identified for 11 testable NFRs. Furthermore, regarding the testing process, we did not observe any testing approach covering the test planning phase. Conclusion: Despite their importance, many testable NFRs seem not be tested due to the lack of appropriate software testing approaches yet. Also, the existing testing approaches do not cover all testing processes activities and, in general, lack empirical evidence about their feasibility and performance, making their use in software projects risky.

No abstract