Assessing Android Test Data Generation Tools via Mutation Testing

Silva, Henrique Neves da; Farah, Paulo Roberto; Mendonça, Willian D. F.; Vergílio, Silvia Regina

doi:10.1145/3356317.3356320

Cited by 3 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of these apps are obtained from the benchmarks: Androtest [25,39], Dynodroid [1,36,37] and EvoDroid [1,36,37], as well as websites like F-Droid [24,42], GitHub [33,34] and Google Play [38]. All apps used in the selected studies are listed in our repository [31].…”

Section: Evaluation Aspectsmentioning

confidence: 99%

“…The second group involves test frameworks for automating the execution of GUI tests. Robotium is mentioned in seven studies, while Espresso and uiautomator are cited in two studies [24,39]. Other tools cited (last group) are Apktool used to reverse engineer APK files, and App Sensorium used to measure energy consumption.…”

Section: Auxiliary Toolsmentioning

confidence: 99%

“…We also analysed the main threats mentioned by the authors in the studies. Only nine studies [1,13,24,25,34,36,37,39,41] mention some threats. Considering these studies, we observe that representativeness of mobile apps appears in 100% of the studies.…”

Section: Main Threatsmentioning

confidence: 99%

“…Espresso 6% [24] uiautomator 6% [39] OTHERS Apktool 6% [33] App Sensorium 6% [24] [39], and Grubbs 8 [24]. Through this analysis, we observed a lack of statistical tests applied to evaluate the results.…”

Section: Statistical Testsmentioning

confidence: 99%

“…The focus of almost all studies is to contribute to the mutation testing of mobile apps. Few studies have targeted the use of mutation testing for test set assessment [13,33,39], as the tool support was initially lacking. There is evidence that the existing tools herein identified are robust and tested in several apps, so they may pave the way for more empirical studies in which mutation testing is applied to assess the quality of test sets.…”

Section: Study Focus and Typesmentioning

confidence: 99%

See 4 more Smart Citations

A mapping study on mutation testing for mobile applications

Silva

Lima

Vergílio

et al. 2021

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Summary The use of mutation testing for mobile applications (apps for short) is still a challenge. Mobile apps are usually event‐driven and encompass graphical user interfaces (GUIs) and a complex execution environment. Then, they require mutant operators to describe specific apps faults, and the automation of the mutation process phases like execution and analysis of the mutants is not an easy task. To encourage research addressing such challenges, this paper presents results from a mapping study on mutation testing for mobile apps. Following a systematic plan, we found 16 primary studies that were analysed according to three aspects: (i) trends and statistics about the field; (ii) study characteristics such as focus, proposed operators and automated support for the mutation testing phases; and (iii) evaluation aspects. The great majority of studies (98%) have been published in the last 3 years. The most addressed language is Java, and Android is the only operating system considered. Mutant operators of GUI and configuration types are prevalent in a total of 138 operators found. Most studies implement a supporting tool, but few tools support mutant execution and analysis. The evaluation conducted by the studies includes apps mainly from the finance and utility domain. Nevertheless, there is a lack of benchmarks and more rigorous experiments. Future research should address other specific types of faults, languages, and operating systems. They should offer support for mutant execution and analysis, as well as to reduce the mutation testing cost and limitations in the mobile context.

show abstract

Section: Evaluation Aspectsmentioning

confidence: 99%

Section: Auxiliary Toolsmentioning

confidence: 99%

Section: Main Threatsmentioning

confidence: 99%

Section: Statistical Testsmentioning

confidence: 99%

Section: Study Focus and Typesmentioning

confidence: 99%

See 3 more Smart Citations

A mapping study on mutation testing for mobile applications

Silva

Lima

Vergílio

et al. 2021

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

show abstract

On the Relation between Code Elements and Accessibility Issues in Android Apps

Silva

Endo

Eler

et al. 2020

Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing

View full text Add to dashboard Cite

Automatic Classification of Equivalent Mutants in Mutation Testing of Android Applications

et al. 2022

View full text Add to dashboard Cite

Software and symmetric testing methodologies are primarily used in detecting software defects, but these testing methodologies need to be optimized to mitigate the wasting of resources. As mobile applications are becoming more prevalent in recent times, the need to have mobile applications that satisfy software quality through testing cannot be overemphasized. Testing suites and software quality assurance techniques have also become prevalent, which underscores the need to evaluate the efficacy of these tools in the testing of the applications. Mutation testing is one such technique, which is the process of injecting small changes into the software under test (SUT), thereby creating mutants. These mutants are then tested using mutation testing techniques alongside the SUT to determine the effectiveness of test suites through mutation scoring. Although mutation testing is effective, the cost of implementing it, due to the problem of equivalent mutants, is very high. Many research works gave varying solutions to this problem, but none used a standardized dataset. In this research work, we employed a standard mutant dataset tool called MutantBench to generate our data. Subsequently, an Abstract Syntax Tree (AST) was used in conjunction with a tree-based convolutional neural network (TBCNN) as our deep learning model to automate the classification of the equivalent mutants to reduce the cost of mutation testing in software testing of android applications. The result shows that the proposed model produces a good accuracy rate of 94%, as well as other performance metrics such as recall (96%), precision (89%), F1-score (92%), and Matthew’s correlation coefficients (88%) with fewer False Negatives and False Positives during testing, which is significant as it implies that there is a decrease in the risk of misclassification.

show abstract

Assessing Android Test Data Generation Tools via Mutation Testing

Cited by 3 publications

References 15 publications

A mapping study on mutation testing for mobile applications

A mapping study on mutation testing for mobile applications

On the Relation between Code Elements and Accessibility Issues in Android Apps

Automatic Classification of Equivalent Mutants in Mutation Testing of Android Applications

Contact Info

Product

Resources

About