Evolving a Test Oracle in Black-Box Testing

Abstract

Abstract. Software testing is an important and expensive activity to the software industry, with testing accounting for over 50% of the cost of software. To ease this problem, test automation is very critical to the process of software testing. One important issue in this automation is to automatically determine whether a program under test (PUT) responds the correct(expected) output for an arbitrary input. In this paper, we model PUTs in black-box way, i.e. processing and responding a list of numbers, and design input/output list relation language(IOLRL) to formally describe the relations between the input and output lists. Given several labelled test cases(test verdicts are set), we use genetic programming to evolve the most distinguishing relations of these test cases in IOLRL and encode the test cases into bit patterns to build a classifier with support vector machine as the constructed test oracle. This classifier can be used to automatically verify if a program output list is the expected one in processing a program input list. The main contribution of this work are the designed IOLRL and the approach to construct test oracle with evolve relations in IOLRL. The experiments show the constructed test oracle has good performance even when few labelled test cases are supplied.

A Novel Learning Algorithm for Büchi Automata Based on Family of DFAs and Classification Trees

“…Learning algorithms are applied to construct environmental assumptions of components in the rule automatically. For the latter, automata learning has been used to automatically generate interface model of computer programs [5,22,26,36,40], a model of system error traces for diagnosis purpose [16], behavior model of programs for statistical program analysis [18], and model-based testing and verification [24,34,39].…”

“…Since the last decade, learning-based automata inference techniques [7,11,30,35] have received significant attention from the community of formal system analysis. In general, the primary applications of automata learning in the community can be categorized into two: improving efficiency and scalability of verification [6,15,17,19,21,23,25,32] and synthesizing abstract system model for further analysis [1,5,16,18,22,24,26,34,36,39].…”

Uma abordagem de teste de software para aplicações de realidade virtual utilizando testes metamórficos

“…• oráculos baseados em aprendizado de máquina: aprendizado de máquina é um conjunto de métodos computacionais, que utilizam dados coletados a fim de possibilitar previsões a respeito de determinados eventos (MITCHELL, 1997). Existem diversas técnicas de aprendizado de máquina supervisionado aplicadas como oráculos de teste, como, por exemplo: (i) redes neurais artificiais (MAO et al, 2006), (ii) máquinas de vetores suporte (WANG et al, 2011) e (iii) redes de inferência fuzzy (AGARWAL et al, 2012);…”