Software Testing Research: Achievements, Challenges, Dreams

Bertolino, Antonia

doi:10.1109/fose.2007.25

Cited by 621 publications

(458 citation statements)

References 78 publications

Supporting

Mentioning

393

Contrasting

Unclassified

Order By: Relevance

“…Code coverage criteria occupy a large part of software testing literature (Ammann and Offutt 2016;Bertolino 2007;Zhu et al 1997). Many authors aim at finding novel criteria that subsume existing ones or that improve fault-finding effectiveness.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing

Miranda

Bertolino²

2017

Software Qual J

Self Cite

View full text Add to dashboard Cite

While the relation between code coverage measures and fault detection is actively studied, only few works have investigated the correlation between measures of coverage and of reliability. In this work, we introduce a novel approach to measuring code coverage, called the operational coverage, that takes into account how much the program's entities are exercised so to reflect the profile of usage into the measure of coverage. Operational coverage is proposed as (i) an adequacy criterion, i.e., to assess the thoroughness of a black box test suite derived from the operational profile, and as (ii) a selection criterion, i.e., to select test cases for operational profile-based testing. Our empirical evaluation showed that operational coverage is better correlated than traditional coverage with the probability that the next test case derived according to the user's profile will not fail. This result suggests that our approach could provide a good stopping rule for operational profile-based testing. With respect to test case selection, our investigations revealed that operational coverage outperformed the traditional one in terms of test suite size and fault detection capability when we look at the average results.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Software testing can be based on many criteria (Ammann and Offutt 2016;Bertolino 2007;Zhu et al 1997). Among them, operational profile-based testing andcoverage-based testing provide two quite diverse approaches.…”

Section: Introductionmentioning

confidence: 99%

An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing

Miranda

Bertolino²

2017

Software Qual J

Self Cite

View full text Add to dashboard Cite

show abstract

“…To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. effectiveness of test data selection criteria and methods of generating test data [3,7,9], considerably less work has been done on the effectiveness of test oracles and their impact on the testing process [4,16,11,12]. Consequently, while we know that the choice of test oracle influences the testing process, we have few, if any techniques for selecting the test oracle, and little understanding of how our selection impacts the testing process.…”

Section: Introductionmentioning

confidence: 99%

Better testing through oracle selection (NIER track)

Staats

Whalen

Heimdahl

2011

Proceedings of the 33rd International Conference on Software Engineering

View full text Add to dashboard Cite

In software testing, the test oracle determines if the application under test has performed an execution correctly. In current testing practice and research, significant effort and thought is placed on selecting test inputs, with the selection of test oracles largely neglected. Here, we argue that improvements to the testing process can be made by considering the problem of oracle selection. In particular, we argue that selecting the test oracle and test inputs together to complement one another may yield improvements testing effectiveness. We illustrate this using an example and present selected results from an ongoing study demonstrating the relationship between test suite selection, oracle selection, and fault finding.

show abstract

“…For example, model-driven testing [2] and modelbased interoperability testing [3] highlight the potential. However, these solutions focus on Web Services and require detailed models of the system's interface syntax (using WSDL) and behavior (using BPEL) in order to generate automated tests.…”

Section: Introductionmentioning

confidence: 99%

“…an IoT sensor produces event data according to the NSGI specification, 2 or streamed data content complies with a data format specification uploaded to the HyperCAT catalogue. -Interoperability testing; monitors the interaction between multiple systems to test whether they interoperate with one another, identifying the specific issues to be resolved where there is failure.…”

Section: Introductionmentioning

confidence: 99%

Model-driven interoperability: engineering heterogeneous IoT systems

2015

View full text Add to dashboard Cite

Interoperability remains a significant burden to the developers of Internet of Things systems. This is because resources and APIs are dynamically composed; they are highly heterogeneous in terms of their underlying communication technologies, protocols and data formats, and interoperability tools remain limited to enforcing standardsbased approaches. In this paper, we propose model-based engineering methods to reduce the development effort towards ensuring that complex software systems interoperate with one another. Lightweight interoperability models can be specified in order to monitor and test the execution of running software so that interoperability problems can be quickly identified, and solutions put in place. A graphical model editor and testing tool are also presented to highlight how a visual model improves upon textual specifications. We show using case-studies from the FIWARE Future Internet Service domain that the software framework can support non-expert developers to address interoperability challenges.

show abstract

Software Testing Research: Achievements, Challenges, Dreams

Cited by 621 publications

References 78 publications

An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing

An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing

Better testing through oracle selection (NIER track)

Model-driven interoperability: engineering heterogeneous IoT systems

Contact Info

Product

Resources

About