Repairing GUI Test Suites Using a Genetic Algorithm

Abstract

Abstract-Recent advances in automated functional testing of Graphical User Interfaces (GUIs) rely on deriving graph models that approximate all possible sequences of events that may be executed on the GUI, and then use the graphs to generate test cases (event sequences) that achieve a specified coverage goal. However, because these models are only approximations of the actual event flows, the generated test cases may suffer from problems of infeasibility, i.e., some events may not be available for execution causing the test case to terminate prematurely. In this paper we develop a method to automatically repair GUI test suites, generating new test cases that are feasible. We use a genetic algorithm to evolve new test cases that increase our test suite's coverage while avoiding infeasible sequences. We experiment with this algorithm on a set of synthetic programs containing different types of constraints and for test sequences of varying lengths. Our results suggest that we can generate new test cases to cover most of the feasible coverage and that the genetic algorithm outperforms a random algorithm trying to achieve the same goal in almost all cases.

Automatically repairing broken workflows for evolving GUI applications

“…Therefore, a tiny UI change like switching the positions of two neighboring menu items can break a GUI test script, but won't affect a workflow from an end-user's perspective. Due to these difficulties, existing test script repairing techniques [14,17,26] fix "execution errors": they identify statements affected by GUI changes or delete unusable actions to make an obsolete test script executable, while ignoring its original semantics. By contrast, FlowFixer aims to preserve the semantics of a broken workflow.…”

“…The 6 workflows that REST can repair all share the same root cause: a menu item is moved from one menu to another without changing its label. We did not compare FlowFixer with other related GUI testing [27,39,40] and GUI test script repairing approaches [17,26] because these approaches target a rather different problem than FlowFixer. Automated GUI testing frameworks like GUITAR [27,39,40] aim to systematically validate a GUI application's functionality.…”

“…Automated GUI testing frameworks like GUITAR [27,39,40] aim to systematically validate a GUI application's functionality. Test script repairing approaches like [17,26] aim to repair unusable test scripts by simply removing or updating affected statements without preserving the original testing semantics, and thus are inapplicable to broken workflow repair. By contrast, FlowFixer recommends replacement UI actions to complete the same workflow.…”

“…To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ISSTA '13, July [15][16][17][18][19][20]2013 As an example of real GUI application evolution, Figure 1 shows screenshots corresponding to versions 0.3 and 0.35 of the Crossword project [4]. The GUI changes break an end-user's workflow.…”

Search-based inference of polynomial metamorphic relations

“…Search-based optimization techniques have been widely applied to software testing, including test-suite generation [8,4,56,24,53] and optimization [72,31,43,3,74]. Besides software testing, search-based optimization techniques have also been applied to fault localization [65], program analysis [76], software refactoring [29,30,55], cost estimation [19], project scheduling [1,18], decisions design optimization [10], automated negotiation [17], source code parallelization [57], requirement engineering [27,64], variability management [41], and so on.…”