Multiple objective test set selection for software product line testing

Filho, Helson Luiz Jakubovski; Ferreira, Thiago Nascimento; Vergílio, Silvia Regina

doi:10.1145/3266237.3266275

Cited by 5 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their approach considers three factors-pairwise coverage, mutation score, and cost-and generates a MOEA using crossover and mutation operators tuned to the feature model being considered. Filho et al extended this work [28,59] to Preference-Based Evolutionary Multi-objective Algorithms, which consider user preferences during the search.…”

Section: Hyperheuristics In Search-based Software Testingmentioning

confidence: 99%

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Almulla

2020

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

Search-based test generation is guided by feedback from one or more fitness functions-scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals-such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage-do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of real Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show small improvements on the third when the number of generations of evolution is fixed. Additionally, for all goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows Evo-SuiteFIT to make strategic choices that efficiently produce more effective test suites, and examining its choices offers insight into how to attain our testing goals. We find that AFFS is a powerful technique to apply when an effective fitness function does not already exist for generating tests to achieve a testing goal.

show abstract

Section: Hyperheuristics In Search-based Software Testingmentioning

confidence: 99%

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Almulla

2020

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

show abstract

“…The HLH was a wellknown hyper-heuristic framework referred to as HyFlex, and the LLHs were 13 common perturbation heuristics and one of movement. Jakubovski-Filho et al (2018a, 2018b proposed two hyper-heuristics based on reference points for software product line testing. These studies used a variant of NSGA-II as HLH and a set of 12 operators of crossover and mutation as LLHs.…”

Section: Introductionmentioning

confidence: 99%

An ACO-based Hyper-heuristic for Sequencing Many-objective Evolutionary Algorithms that Consider Different Ways to Incorporate the DM's Preferences

Rivera

Cruz-Reyes²,

Fernández³

et al. 2023

Swarm and Evolutionary Computation

View full text Add to dashboard Cite

Learning how to search: generating effective test cases through adaptive fitness function selection

Almulla

2022

Empir Software Eng

View full text Add to dashboard Cite

Search-based test generation is guided by feedback from one or more fitness functions—scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals—such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage—do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show limited improvements on the third when the number of generations of evolution is fixed. Additionally, for two of the three goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows strategic choices that efficiently produce more effective test suites, and examining these choices offers insight into how to attain our testing goals. We find that adaptive fitness function selection is a powerful technique to apply when an effective fitness function does not already exist for achieving a testing goal.

show abstract

Multiple objective test set selection for software product line testing

Cited by 5 publications

References 28 publications

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

An ACO-based Hyper-heuristic for Sequencing Many-objective Evolutionary Algorithms that Consider Different Ways to Incorporate the DM's Preferences

Learning how to search: generating effective test cases through adaptive fitness function selection

Contact Info

Product

Resources

About