Enabling mutation testing for Android apps

Linares‐Vásquez, Mario; Bavota, Gabriele; Tufano, Michele; Moran, Kevin; Penta, Massimiliano Di; Vendome, Christopher; Bernal-Cárdenas, Carlos; Poshyvanyk, Denys

doi:10.1145/3106237.3106275

Cited by 71 publications

(56 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Linares-Vásquez et al [52] also investigated android app bugs very recently, but our study significantly differs from theirs. We focuses on framework exceptions and give a comprehensive, deep analysis, including exception manifestations, root causes, abilities of existing bug analysis tools, and fixing practices.…”

Section: Discussionmentioning

confidence: 66%

Large-scale analysis of framework-specific exceptions in Android apps

Fan

Chen

et al. 2018

Proceedings of the 40th International Conference on Software Engineering

View full text Add to dashboard Cite

Mobile apps have become ubiquitous. For app developers, it is a key priority to ensure their apps' correctness and reliability. However, many apps still suffer from occasional to frequent crashes, weakening their competitive edge. Large-scale, deep analyses of the characteristics of real-world app crashes can provide useful insights to guide developers, or help improve testing and analysis tools. However, such studies do not exist -this paper fills this gap. Over a four-month long effort, we have collected 16,245 unique exception traces from 2,486 open-source Android apps, and observed that framework-specific exceptions account for the majority of these crashes. We then extensively investigated the 8,243 frameworkspecific exceptions (which took six person-months): (1) identifying their characteristics (e.g., manifestation locations, common fault categories), (2) evaluating their manifestation via state-of-the-art bug detection techniques, and (3) reviewing their fixes. Besides the insights they provide, these findings motivate and enable follow-up research on mobile apps, such as bug detection, fault localization and patch generation. In addition, to demonstrate the utility of our findings, we have optimized Stoat, a dynamic testing tool, and implemented ExLocator, an exception localization tool, for Android apps. Stoat is able to quickly uncover three previously-unknown, confirmed/fixed crashes in Gmail and Google+; ExLocator is capable of precisely locating the root causes of identified exceptions in real-world apps. Our substantial dataset is made publicly available to share with and benefit the community. CCS CONCEPTS• Software and its engineering → Software testing and debugging; * Ting Su, Lihua Xu and Geguang Pu are the corresponding authors. Lingling Fan and Ting Su contributed equally to this work.

show abstract

Section: Discussionmentioning

confidence: 66%

Large-scale analysis of framework-specific exceptions in Android apps

Fan

Chen

et al. 2018

Proceedings of the 40th International Conference on Software Engineering

View full text Add to dashboard Cite

show abstract

“…Are they common to other GUI frameworks? To answer this, we conducted a thorough and careful inspection on (1) Android docs and APIs [23], including the principle of single-GUIthread model [40], various async programming constructs [27, 32-34, 37, 38, 41, 42], GUI components [21,28,30,36,43], etc; and (2) technical posts filtered from Stack Overflow (the largest developer Q&A community) by the keywords "Android" plus the names of async constructs, tutorials on async programming [13]; and (3) fault studies on Android [18,46,55,86]. Answer: We identified 3 async programming rules ( Fig.…”

Section: Formative Studymentioning

confidence: 99%

Efficiently manifesting asynchronous programming errors in Android apps

Fan

Sus

Chen

et al. 2018

Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

View full text Add to dashboard Cite

Android, the #1 mobile app framework, enforces the single-GUIthread model, in which a single UI thread manages GUI rendering and event dispatching. Due to this model, it is vital to avoid blocking the UI thread for responsiveness. One common practice is to offload long-running tasks into async threads. To achieve this, Android provides various async programming constructs, and leaves developers themselves to obey the rules implied by the model. However, as our study reveals, more than 25% apps violate these rules and introduce hard-to-detect, fail-stop errors, which we term as aysnc programming errors (APEs). To this end, this paper introduces APEChecker, a technique to automatically and efficiently manifest APEs. The key idea is to characterize APEs as specific fault patterns, and synergistically combine static analysis and dynamic UI exploration to detect and verify such errors. Among the 40 real-world Android apps, APEChecker unveils and processes 61 APEs, of which 51 are confirmed (83.6% hit rate). Specifically, APEChecker detects 3X more APEs than the state-of-art testing tools (Monkey, Sapienz and Stoat), and reduces testing time from half an hour to a few minutes. On a specific type of APEs, APEChecker confirms 5X more errors than the data race detection tool, EventRacer, with very few false alarms. CCS CONCEPTS• Software and its engineering → Software testing and debugging;such fatal programming errors that violate the rules implied by the single-UI-thread model as async programming errors (APEs).Such bugs in Android are not easy to detect manually, due to (1) they usually reside in the code of handling interactions between UI thread and async threads, which can be rather complicated for manual analysis; (2) they can only be triggered at the right states of GUI components (e.g., activity, fragment) with complicated lifecycle [21,30]; (3) they have to be triggered at right thread scheduling, while the execution time of async threads is affected by the task and its running environment (e.g., network stability, system load).Even worse, existing bug detection techniques are ineffective for such bugs. First, most GUI testing techniques, e.g., random testing [39,57], search-based testing [58,60], and model-based testing [2,3,8,78,85], are designed for functional testing in general. They aim at enumerating all possible event sequences (GUI-level events in particular) to manifest bugs, which is unscalable and time-consuming. Additionally, they mainly aim at improving code coverage, which may not be sufficient for exhibiting APEs -require specific event sequences with appropriate lifecycle states and thread scheduling. Second, static analysis tools, e.g., Lint [35], Find-Bugs [19] and PMD [67], although scalable, only enforce simple rules (syntax or trivial control/data-flow analysis) to locate suspicious bugs. For example, Lint declares it can find "WrongThread" errors (one type of APEs) [24]. However, as our evaluation in Section 5 demonstrates, Lint incurs a number of false negatives -failing to detect those so...

show abstract

“…In order to ensure that MDroid+ is an effective, practical, and flexible/extensible tool for mutation testing, it takes into account the following design considerations: (i) an empirically derived set of mutation operators; (ii) a design embracing the open/closed principle (i.e., open to extension, closed to modification); (iii) visitor and factory design patterns for deriving the Potential Failure Profile (PFP) and applying operators, (iv) parallel computation for efficient mutant seeding. MDroid+ is written in Java and available as an open source project [18]. In the following sections, we describe MDroid+ according to its workflow described in Figure 1.…”

Section: Approachmentioning

confidence: 99%

“…In this paper, we consider non-compilable mutants as those that are syntactically incorrect and cause compilation/assembly errors, and trivial mutants as those that are killed arbitrarily by most test cases (e.g., crashing on launch). The trivial mutant study was supported by a large-scale dynamic analysis framework [17]. Figure 2 reports the results of (i) the percentage of non-compilable mutants (NCM), (ii) the percentage of trivial mutants (TM), and (iii) the total number of generated mutants per app.…”

Section: Evaluation 31 Study Contextmentioning

confidence: 99%

“…In this paper, we describe MDroid+, a mutation testing framework for Android apps that aims to support developers in writing mobile tests. MDroid+ includes 38 Android and Java specific mutation operators that were designed according to an empirically arXiv:1802.04749v1 [cs.SE] 13 Feb 2018 derived taxonomy of common, naturally occurring faults in Android applications [17]. The main contributions of MDroid+ can be summarized as follows:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

MDroid+

Moran

Tufano

Bernal-Cárdenas

et al. 2018

Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Self Cite

View full text Add to dashboard Cite

Mutation testing has shown great promise in assessing the effectiveness of test suites while exhibiting additional applications to test-case generation, selection, and prioritization. Traditional mutation testing typically utilizes a set of simple language specific source code transformations, called operators, to introduce faults. However, empirical studies have shown that for mutation testing to be most effective, these simple operators must be augmented with operators specific to the domain of the software under test. One challenging software domain for the application of mutation testing is that of mobile apps. While mobile devices and accompanying apps have become a mainstay of modern computing, the frameworks and patterns utilized in their development make testing and verification particularly difficult. As a step toward helping to measure and ensure the effectiveness of mobile testing practices, we introduce MDroid+, an automated framework for mutation testing of Android apps. MDroid+ includes 38 mutation operators from ten empirically derived types of Android faults and has been applied to generate over 8,000 mutants for more than 50 apps. Video URL: https://youtu.be/yzE5_-zN5GA CCS CONCEPTS• Software and its engineering → Software verification and validation;

show abstract

Enabling mutation testing for Android apps

Cited by 71 publications

References 74 publications

Large-scale analysis of framework-specific exceptions in Android apps

Large-scale analysis of framework-specific exceptions in Android apps

Efficiently manifesting asynchronous programming errors in Android apps

MDroid+

Contact Info

Product

Resources

About