Understanding the Test Automation Culture of App Developers

Kochhar, Pavneet Singh; Thung, Ferdian; Nagappan, Nachiappan; Zimmermann, Thomas; Lo, David

doi:10.1109/icst.2015.7102609

Cited by 125 publications

(84 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The good news is that we observe an increasing adoption of unit and UI tests in the last two years. This trend can be observed by comparing our findings with previous work (Kochhar et al, 2015); while the previous study highlights that the prevalence of automated tests in mobile apps was merely 14%, in this work, we observe that 41% of FOSS apps are developed with automated testing tools.…”

Section: All Testssupporting

confidence: 81%

“…We analyze the adoption of unit tests, Graphical User Interface (GUI) tests, cloud based testing services, and Continuous Integration / Continuous Deployment (CI/CD). Previous work, in a survey with 83 Android developers, suggests that mobile developers are failing to adopt automated testing techniques (Kochhar et al, 2015). This is concerning since testing is an important factor in software maintainability (Visser et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

To the attention of mobile software developers: guess what, test your app!

Cruz

Abreu

2019

Empir Software Eng

Self Cite

View full text Add to dashboard Cite

Software testing is an important phase in the software development lifecycle because it helps in identifying bugs in a software system before it is shipped into the hand of its end users. There are numerous studies on how developers test general-purpose software applications. The idiosyncrasies of mobile software applications, however, set mobile apps apart from general-purpose systems (e.g., desktop, stand-alone applications, web services). This paper investigates working habits and challenges of mobile software developers with respect to testing. A key finding of our exhaustive study, using 1000 Android apps, demonstrates that mobile apps are still tested in a very ad hoc way, if tested at all. However, we show that, as in other types of software, testing increases the quality of apps (demonstrated in user ratings and number of code issues). Furthermore, we find evidence that tests are essential when it comes to engaging the community to contribute to mobile open source software. We discuss reasons and potential directions to address our findings. Yet another relevant finding of our study is that Continuous Integration and Continuous Deployment (CI/CD) pipelines are rare in the mobile apps world (only 26% of the apps are developed in projects employing CI/CD) -we argue that one of the main reasons is due to the lack of exhaustive and automatic testing.

show abstract

Section: All Testssupporting

confidence: 81%

Section: Introductionmentioning

confidence: 99%

To the attention of mobile software developers: guess what, test your app!

Cruz

Abreu

2019

Empir Software Eng

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hence, an in-depth testing of their GUIs (i.e., Graphical User Interfaces), through which most of the features are exposed, becomes fundamental. Specialized literature has however highlighted a scarce adoption of testing tools among all categories of mobile applications [9] [16]. Among the main difficulties discouraging mobile testing, mobile apps are particularly prone to GUI testing Fragility, due to a rapid evolution of their GUI.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Fragility of layout-based and visual GUI test scripts: an assessment study on a hybrid mobile application

Coppola

Ardito

Torchiano

2019

Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

View full text Add to dashboard Cite

Context: Albeit different approaches exist for automated GUI testing of hybrid mobile applications, the practice appears to be not so commonly adopted by developers. A possible reason for such a low diffusion can be the fragility of the techniques, i.e. the frequent need for maintaining test cases when the GUI of the app is changed.Goal: In this paper, we perform an assessment of the maintenance needed by test cases for a hybrid mobile app, and the related fragility causes.Methods: We evaluated a small test suite with a Layout-based testing tool (Appium) and a Visual one (EyeAutomate) and observed the changes needed by tests during the co-evolution with the GUI of the app.Results: We found that 20% Layout-based test methods and 30% Visual test methods had to be modified at least once, and that each release induced fragilities in 3-4% of the test methods.Conclusion: Fragility of GUI tests can induce relevant maintenance efforts in test suites of large applications. Several principal causes for fragilities have been identified for the tested hybrid application, and guidelines for developers are deduced from them. CCS CONCEPTS• Software and its engineering → Software verification and validation; Software testing and debugging;

show abstract

“…However, the majority of mobile applications are developed within restricted resources, budget, and limited time due to time‐to‐market pressure. This severely affects the quality of the mobile applications development . Therefore, testing mobile applications adequately within a constricted time and cost becomes a big challenge that will affect the continuality of the mobile applications development .…”

Section: Introductionmentioning

confidence: 99%

Automated parallel GUI testing as a service for mobile applications

Ali

Maghawry

Badr

2018

J Software Evolu Process

View full text Add to dashboard Cite

Recently, testing mobile applications is gaining much attention due to the widespread of smartphones and the tremendous number of mobile applications development. It is essential to test mobile applications before being released for the public use. Graphical user interface (GUI) testing is a type of mobile applications testing conducted to ensure the proper functionality of the GUI components. Typically, GUI testing requires a lot of effort and time whether manual or automatic. Cloud computing is an emerging technology that can be used in the software engineering field to overcome the defects of the traditional testing approaches by using cloud computing resources. As a result, testing‐as‐a‐service is introduced as a service model that conducts all testing activities in a fully automated manner. In this paper, a system for mobile applications GUI testing based on testing‐as‐a‐service architecture is proposed. The proposed system performs all testing activities including automatic test case generation and simultaneous test execution on multiple virtual nodes for testing Android‐based applications. The proposed system reduces testing time and meets fast time‐to market constraint of mobile applications. Moreover, the proposed system architecture addresses many issues such as maximizing resource utilization, continuous monitoring to ensure system reliability, and applying fault‐tolerance approach to handle occurrence of any failure.

show abstract

Understanding the Test Automation Culture of App Developers

Cited by 125 publications

References 16 publications

To the attention of mobile software developers: guess what, test your app!

To the attention of mobile software developers: guess what, test your app!

Fragility of layout-based and visual GUI test scripts: an assessment study on a hybrid mobile application

Automated parallel GUI testing as a service for mobile applications

Contact Info

Product

Resources

About