Systematic execution of Android test suites in adverse conditions

Adamsen, Christoffer Quist; Mezzetti, Gianluca; Møller, Anders

doi:10.1145/2771783.2771786

Cited by 92 publications

(39 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most of the failures detected by THOR belong to GUI category. The results achieved both by Zaeem et al [15] and Adamsen et al [12] gave us a hint about the relevance of the problem addressed in our paper, since several failures discovered by their techniques were GUI-related and exposed by the orientation change.…”

Section: Testing Apps Through Mobile Specific Eventssupporting

confidence: 52%

See 1 more Smart Citation

Why does the orientation change mess up my Android application? From GUI failures to code faults

Amalfitano

Riccio

Paiva

et al. 2017

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryThis paper investigates the failures exposed in mobile apps by the mobile-specific event of changing the screen orientation. We focus on GUI failures resulting in unexpected GUI states that should be avoided to improve the apps quality and to ensure better user experience. We propose a classification framework that distinguishes 3 main classes of GUI failures due to orientation changes and exploit it in 2 studies that investigate the impact of such failures in Android apps. The studies involved both open-source and apps from Google Play that were specifically tested exposing them to orientation change events. The results showed that more than 88% of these apps were affected by GUI failures, some classes of GUI failures were more common than others, and some GUI objects were more frequently involved. The app source code analysis allowed us to identify 6 classes of common faults causing specific GUI failures. KEYWORDSAndroid bugs, Android testing, GUI failures, GUI testing, mobile testing, orientation change INTRODUCTIONOver the last decade, the number of users of mobile technology and smartphones has increased considerably. The total number of smartphone users worldwide is forecast to surpass 2.5 billion in 2019 [1].This causes a constant demand for new software applications running on these devices (apps). As of the month of June 2016, both Android and iOS users had the opportunity to choose from among more than 2 million apps [2].Mobile technology has radically changed the lifestyle of billions of people around the world. We use mobile apps for several hours every day, entrust them our sensitive data, and perform a large variety of activities through them, including critical tasks.The demand for app quality has grown together with their spread. Apps users require them to be reliable, robust, efficient, secure, usable, etc.As a consequence, software developers should give proper consideration to the quality of their applications by adopting suitable quality assurance techniques, such as testing.In the last decade, the research community has devoted great interest to the mobile app testing field. Several testing approaches have been proposed to assess different quality aspects of mobile applications [3], such as functionality [4], performance [5], security [6,7], responsiveness [8], and energy consumption [9].Since mobile apps are event-driven systems, many proposed techniques solicit them by means of sequences of events [10]. However, because of the peculiarities of the mobile devices, these apps should be tested with appositely crafted approaches [11]. As an example, testing processes should devote particular attention to exercise the apps through mobile-specific events, such as sending an application to the background and resuming it, receiving a call, changing the state of the network connections, or changing the orientation of the device.Among these types of events, the orientation change deserves special attention. It is a peculiar event in mobile platforms that causes the switch of the running app b...

show abstract

Section: Testing Apps Through Mobile Specific Eventssupporting

confidence: 52%

“…* Unfortunately, the implementation of these recommendations is not straightforward and introduces programming challenges to Android programmers. Several works in the literature have pointed out that many mobile apps actually crash or show failures that can be attributed to orientation change mishandling [4,12,13,14,15].…”

Section: Introductionmentioning

confidence: 99%

Why does the orientation change mess up my Android application? From GUI failures to code faults

Amalfitano

Riccio

Paiva

et al. 2017

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…In the third case, the app correctly handles stop-start events, but its behavior is broken by an upgrade of the Android framework, such as in the PagerSlidingTabStrip that can loose the color selected for its ActionBar after the upgrade to API 17 [5]. Introducing any of these problems is relatively easy, as also reported in other studies [6], [7].…”

Section: Introductionmentioning

confidence: 93%

“…While there are several testing solutions for Android, such as Robotium [8], Thor [7], and AppDoctor [6], none of these provide specific support to data loss problems detection. Robotium [8] simply offers an environment that testers can exploit to implement capture and replay testing.…”

Section: Introductionmentioning

confidence: 99%

Healing Data Loss Problems in Android Apps

Riganelli

Micucci

Mariani

2016

2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)

View full text Add to dashboard Cite

Android apps should be designed to cope with stop-start events, which are the events that require stopping and restoring the execution of an app while leaving its state unaltered. These events can be caused by run-time configuration changes, such as a screen rotation, and by context-switches, such as a switch from one app to another. When a stop-start event occurs, Android saves the state of the app, handles the event, and finally restores the saved state. To let Android save and restore the state correctly, apps must provide the appropriate support. Unfortunately, Android developers often implement this support incorrectly, or do not implement it at all. This bad practice makes apps to incorrectly react to stop-start events, thus generating what we defined data loss problems, that is Android apps that lose user data, behave unexpectedly, and crash due to program variables that lost their values. Data loss problems are difficult to detect because they might be observed only when apps are in specific states and with specific inputs. Covering all the possible cases with testing may require a large number of test cases whose execution must be checked manually to discover whether the app under test has been correctly restored after each stop-start event. It is thus important to complement traditional in-house testing activities with mechanisms that can protect apps as soon as a data loss problem occurs in the field. In this paper we present DataLossHealer, a technique for automatically identifying and healing data loss problems in the field as soon as they occur. DataLossHealer is a technique that checks at run-time whether states are recovered correctly, and heals the app when needed. DataLossHealer can learn from experience, incrementally reducing the overhead that is introduced avoiding to monitor interactions that have been managed correctly by the app in the past.Comment: IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 201

show abstract

“…Adamsen et al [Adamsen et al, 2015] present an approach to improve existing test suites by injecting them with special events that represent adverse conditions (e.g., unexpected events that may arise at any time or memory shortage). The main problem they faced is the decision of when to inject the event while not disrupting the normal functioning of the AUT nor increasing the time of the test suite on a non-scalable manner.…”

Section: Maintainabilitymentioning

confidence: 99%

Automated Pattern-Based Testing of Mobile Applications

Morgado

Paiva

Faria

2014

2014 9th International Conference on the Quality of Information and Communications Technology

View full text Add to dashboard Cite

Testing is essential to improve the quality of a product and is, thus, a crucial part of any development process. However, the amount of time and resources companies dedicate to it is usually reduced, which justifies the high focus of the community on improving testing automation. Considering the importance smartphones have acquired in our daily lives, it is extremely important to define techniques to test mobile applications in order to ensure their quality.This document presents a testing approach and tool (iMPAcT) to improve the automation of mobile testing based on the presence of recurring behaviour, UI Patterns. It combines reverse engineering, pattern matching and testing. The reverse engineering process is responsible for crawling the application, i.e. to analyse the state of the application and to interact with it by firing events. The pattern matching tries to identify the presence of UI patterns based on a catalogue of patterns. When a UI Pattern from the catalogue is detected, a test strategy is applied. These test strategies are called UI Test Patterns. These three phases work in an iterative way: the patterns and identified and tested between firing of events, i.e. the process alternates between exploring the application and testing the UI Patterns. The process is dynamic and fully automatic not requiring any previous knowledge about the application under test.A catalogue of patterns is presented in this document. It has UI Patterns to identify on mobile applications and UI Test Patterns defining generic test strategies to test them. These patterns are generic and, thus, can be applied to any application.In order to validate the overall approach we have conducted three experiments. The goal is to analyse the capacity of iMPAcT of finding failures and assess if the results are reliable. In addition, we also measured the execution time and events coverage. Este documento apresenta uma abordagem de teste, e respetiva ferramenta (iMPAcT), para melhorar a automação de teste de aplicações móveis baseado na presença de comportamento recorrente, padrões de UI. Esta abordagem combina engenharia reversa, deteção de padrões e teste. O processo de engenharia reversa é responsável pela exploração da aplicação, ou seja, pela análise do estado da aplicação e a interação com ela através de eventos. O processo de deteção de padrões baseia-se num catálogo de padrões para identificar padrões de UI. Quando um padrão de UI do catálogo é detetado, uma estratégia de teste é aplicada. Estas estratégias de teste denominam-se padrões de teste. Estas três fases trabalham de forma iterativa: os padrões são identificados e testados entre eventos, ou seja, o processo alterna entre a exploração da aplicação e o teste dos padrões de UI. O processo é dinâmico e completamente automático e não necessita de qualquer conhecimento prévio acerca da aplicação a testar.Um catálogo de padrões é apresentado neste documento. Contém padrões de UI para serem identificados em aplicações móveis e padrões de teste que definem estratégias ...

show abstract

Systematic execution of Android test suites in adverse conditions

Cited by 92 publications

References 29 publications

Why does the orientation change mess up my Android application? From GUI failures to code faults

Why does the orientation change mess up my Android application? From GUI failures to code faults

Healing Data Loss Problems in Android Apps

Automated Pattern-Based Testing of Mobile Applications

Contact Info

Product

Resources

About