Tester interactivity makes a difference in search-based software testing: A controlled experiment

Marculescu, Bogdan; Poulding, Simon; Feldt, Robert; Petersen, Kai; Torkar, Richard

doi:10.1016/j.infsof.2016.05.009

Cited by 13 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The changes to the ISBST system required us to go through the validation in the laboratory and static validation steps again. The additional validations used lessons learned from previous versions and focused on the effect of interaction on the search process [4], and on investigating the use of exploration to augment the ISBST system [6]. These efforts, however, validate the updated ISBST system in the laboratory, in an academic setting.…”

Section: The Development and Previous Evaluations Of The Isbst Systemmentioning

confidence: 99%

“…One example of developing and evaluating new approaches is our work with the Interactive Search-Based Software Testing (ISBST) system. The ISBST system was proposed [3], was evaluated both in academia [4] and in industry [5], and further refinements have been proposed [6]. Thus, the ISBST system has been evaluated and validated in academia, and preparations for its transfer to industry are ongoing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transferring interactive search-based software testing to industry

Marculescu

Feldt

Torkar

et al. 2018

Journal of Systems and Software

Self Cite

View full text Add to dashboard Cite

Context: Search-Based Software Testing (SBST), and the wider area of Search-Based Software Engineering (SBSE), is the application of optimization algorithms to problems in software testing, and software engineering, respectively. New algorithms, methods, and tools are being developed and validated on benchmark problems. In previous work, we have also implemented and evaluated Interactive Search-Based Software Testing (ISBST) tool prototypes, with a goal to successfully transfer the technique to industry.Objective: While SBST and SBSE solutions are often validated on benchmark problems, there is a need to validate them in an operational setting, and to assess their performance in practice. The present paper discusses the development and deployment of SBST tools for use in industry, and reflects on the transfer of these techniques to industry.Method: In addition to previous work discussing the development and validation of an ISBST prototype, a new version of the prototype ISBST system was evaluated in the laboratory and in industry. This evaluation is based on an industrial System under Test (SUT) and was carried out with industrial practitioners. The Technology Transfer Model is used as a framework to describe the progression of the development and evaluation of the ISBST system, as it progresses through the first five of its seven steps.Results: The paper presents a synthesis of previous work developing and evaluating the ISBST prototype, as well as presenting an evaluation, in both academia and industry, of that prototype's latest version. In addition to the evaluation, the paper also discusses the lessons learned from this transfer.Conclusions: This paper presents an overview of the development and deployment of the ISBST system in an industrial setting, using the framework of the Technology Transfer Model. We conclude that the ISBST system is capable of evolving useful test cases for that setting, though improvements in the means the system uses to communicate that information to the user are still required. In addition, a set of lessons learned from the project are listed and discussed. Our objective is to help other researchers that wish to validate searchbased systems in industry, and provide more information about the benefits and drawbacks of these systems.

show abstract

Section: The Development and Previous Evaluations Of The Isbst Systemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Transferring interactive search-based software testing to industry

Marculescu

Feldt

Torkar

et al. 2018

Journal of Systems and Software

Self Cite

View full text Add to dashboard Cite

show abstract

“…En Suecia, en el Instituto de Tecnología Karlskrona y la Universidad de Gotemburgo, se realiza una investigación en la "interactividad de las pruebas de software basado en búsquedas bajo un experimento controlado" [11]. Aquí se realiza un proceso de investigación y seguimiento riguroso al sistema de búsqueda interactivo basado en software de testing (isbst) a través de un experimento controlado con 58 estudiantes que cursan la asignatura Verificación y validación de software.…”

Section: En El Contexto Mundialunclassified

Análisis del proceso de pruebas de calidad de software

Paz

2016

ing. Solidar

View full text Add to dashboard Cite

Introducción: este artículo es producto de la lectura, revisión y análisis de libros, revistas y artículos reconocidos por su calidad científica e investigativa que han abordado el proceso de pruebas de calidad de software. El autor, con base en su experiencia laboral en empresas de desarrollo de software, docencia y otras, ha recopilado y seleccionado información para argumentar y sustentar el porqué de la importancia del proceso de pruebas de calidad de software. Metodología: se analizó la literatura existente sobre el proceso de pruebas de calidad de software en un contexto local, nacional y mundial. Se revisaron de forma exhaustiva las bases de datos ScienceDirect, Elseiver, Springer, Wiley Online Library, proquest,Enginneering Village, Scopus, Dialnet. Resultados: se describen conceptos de gran importancia en el proceso de pruebas, características, metodologías y marcos de referencia enfocados a la adecuada implementación de un proceso de ruebas. Conclusiones: se generan unos resultados y conclusiones con el fin de que las empresas de desarrollo de productos software mejoren el rendimiento y la efectividad, así como la optimización de los procesos de pruebas de calidad de software, que además es base fundamental para iniciar procesos de investigación en calidad de software.

show abstract

“…In general, two main benefits arise from this perspective, which are to provide meaningful insights to the DM and increase the human engagement [6]. As discussed by Marculescu et al [7], intuitive interaction with domain specialists is a key factor in industrial applicability, since it makes the system more usable and more easily accepted in an industrial setting. Despite the promising outlook, the definition regarding the type and how the preferences are exploited by optimization algorithms is a relevant challenge and has attracted attention in recent years.…”

Section: Introductionmentioning

confidence: 99%

Incorporating decision maker’s preferences in a multi-objective approach for the software release planning

et al. 2017

View full text Add to dashboard Cite

Background: Release planning (RP) is one of the most complex and relevant activities in the iterative and incremental software development, because it addresses all decisions associated with the selection and assignment of requirements to releases. There are many approaches in which RP is formalized as an optimization problem. In this context, search-based software engineering (SBSE) deals with the application of search techniques to solve complex problems of software engineering. Since RP is a wicked problem with a large focus on human intuition, the decision maker's (DM) opinion is a relevant issue to be considered when solving release planning problem. Thus, we emphasize the importance in gathering the DM's preferences to guide the optimization process through search space area of his/her interests. Methods: Typically, RP is modelled as a multi-objective problem by considering to maximize overall clients satisfaction and minimize project risk. In this paper, we extend this notion and consider DM's preferences as an additional objective. The DM defines a set of preferences about the requirements allocation which is stored in a preference base responsible for influencing the search process. The approach was validated through an empirical study, which consists of two different experiments, respectively identified as (a) automatic experiment and (b) participant-based experiment. Basically, the former aims to analyze the approach using different search-based algorithms (NSGA-II, MOCell, IBEA, and SPEA-II), over artificial and real-world instances, whereas the latter aims at evaluating the use of the proposal in a real scenario composed of human evaluations. Results: The automatic experiment points out that NSGA-II obtained overall superiority in two of the three datasets investigated, positioning itself as a superior search technique for scenarios with few number of requirements and preferences, while IBEA showed to be better for larger ones (with more requirements and preferences). Regarding the participant-based experiment, it was found that two thirds of the participants evaluated the preference-based solution better than the non-preference-based one. Conclusions:The results suggest that it is feasible to investigate the approach in a real-world scenario. In addition, we made available a prototype tool in order to incorporate the human's preferences about the requirements allocation into the solution of release planning.

show abstract

Tester interactivity makes a difference in search-based software testing: A controlled experiment

Cited by 13 publications

References 34 publications

Transferring interactive search-based software testing to industry

Transferring interactive search-based software testing to industry

Análisis del proceso de pruebas de calidad de software

Incorporating decision maker’s preferences in a multi-objective approach for the software release planning

Contact Info

Product

Resources

About