A Taxonomy to Assess and Tailor Risk-Based Testing in Recent Testing Standards

Abstract

This article provides a taxonomy for risk-based testing that serves as a tool to define, tailor, or assess risk-based testing approaches in general and to instantiate risk-based testing approaches for the current testing standards ISO/IEC/IEEE 29119, ETSI EG and OWASP Security Testing Guide in particular. We demonstrate the usefulness of the taxonomy by applying it to the aforementioned standards as well as to the risk-based testing approaches SmartTesting, RACOMAT, PRISMA and riskbased test case prioritization using fuzzy expert systems. In this setting, the taxonomy is used to systematically identify deviations between the standards'requirements and the individual testing approaches so that we are able to position and compare the testing approaches and discuss their potential for practical application.

Un mapeo sistemático de literatura: pruebas basadas en riesgo en el desarrollo de software

“…For example, a taxonomy of RBT provides a framework for understanding approaches to RBT and adapting them to specific purposes by including three types of approaches: risk drivers, risk assessment, and RBT (9) . It has also been possible to find a taxonomy where categorization is made between standards and approaches presented to incorporate RBT (4) . In (14) , a light approach is presented to estimate the risk probability in software testing, using phases: (i) risk elements definition, (ii) probability, (iii) impact estimation, (iv) risk values calculation, (v) risk levels determination, (vi) testing strategy definition, (vii) testing strategy refinement.…”

“…In the time window established and presented in Figure 1, since year 2000 onwards, there is an increasing interest in RBT, with research increasing from 2012 to the present. The percentage of studies according to the classification type per year is: (i) 22.7% corresponding to approaches: 2012 (30) , 2017 (14) , 2018 (22) , 2020 (32,33) ; (ii) 18.2% corresponds to case study: 2000 (21) , 2010 (29) , 2014 (2) , 2016 (12) ; (iii) 9.1% corresponds to Taxonomy: 2014 (9) , 2019 (4) ; (iv) 9.1% corresponds to Techniques: 2005 (26) , 2018 (28) . (v) 40.9% corresponds to one article per year in Framework 2014 (27) , Tools 2007 (31) , Method 2016 (8) , Methodology 2013 (25) , Model 2012 (13) , Prediction of Defects 2016 (23) , Procedure 2014 (7) , Process 2010 (10) and Exploratory Review 2016 (24) .…”

“…A testing type that has started to be incorporated and studied is riskbased testing (RBT), which focuses on testing activities on those areas that trigger the most critical software system situations (2,3) . According to the international standard ISO/IEC/IEEE 29119, risks should be considered as a fundamental part of the testing process (4) , where a risk element in the testing context is any tested value element, for example, a requirement, a component or an error (2) . In this case, when risks are identified they are prioritized according to both their probability and impact, and test cases are projected based on strategies for the identified risk factors treatment (5) .…”

“…Incorporating RBT into software projects from the first stages of product development will allow timely follow-up of testing until it is guaranteed the risk identified in the final product is not affected (4) and optimize the resource allocation such as budget, time and people (6) . On the other hand, it helps to mitigate the risks associated with the product, identifying those critical areas that may require it, and providing support for decision-making in the management of the project (7) .…”

Comparative Analysis of Risk Management Techniques for Large-Scale Systems

“…In addition, established approaches and approaches derived from original research show how to meet the requirements of the standard and go beyond these recommendations. [2] The risks in software projects can also be related to finance. A paper has discussed these risks and proposed a solution using Reference Class Forecasting Technique.…”