Test automation in open-source Android apps

Abstract

Automated testing of mobile apps has received significant attention in recent years from researchers and practitioners alike. In this paper, we report on the largest empirical study to date, aimed at understanding the test automation culture prevalent among mobile app developers. We systematically examined more than 3.5 million repositories on GitHub and identified more than 12, 000 non-trivial and real-world Android apps. We then analyzed these non-trivial apps to investigate (1) the prevalence of adoption of test automation; (2) working habits of mobile app developers in regards to automated testing; and (3) the correlation between the adoption of test automation and the popularity of projects. Among others, we found that (1) only 8% of the mobile app development projects leverage automated testing practices; (2) developers tend to follow the same test automation practices across projects; and (3) popular projects, measured in terms of the number of contributors, stars, and forks on GitHub, are more likely to adopt test automation practices. To understand the rationale behind our observations, we further conducted a survey with 148 professional and experienced developers contributing to the subject apps. Our findings shed light on the current practices and future research directions pertaining to test automation for mobile app development. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

Test automation maturity improves product quality—Quantitative study of open source projects using continuous integration

“…This team reported that, the higher level of test automation maturity is associated with shorter release cycles and better product quality in the CI context. [33] 2020 As noted in Table 2, in all studies [27,33,34,35,28,37,38,39,30,32] that have observed how test automation maturity affects product quality, the findings verified the positive impact. These studies view product quality as to which extent a software product free from defects during the quality assurance process or in the production environment.…”

“…One school of viewpoint states the negative relationship of test automation maturity and product quality with test automation effort. Studies [33,28,37,30,31], which hold this view, have observed the increased effort to develop and execute more rigorous automated tests in attaining higher product quality. Experience study [36] reported that the direct effect of test automation maturity on test automation effort is negative.…”

“…• Our study was recently finished and intended to present the current view for this research scope. As shown in Table 2, most of the prior studies were published several years ago, while Only [27,33] are recent studies. • Our findings evidenced some observations of prior studies, resolved the different viewpoints of prior studies, and identified novel research topics to extend the impact and scope of CI and test automation literature.…”

Empirical Study of Test Case and Test Framework Presence in Public Projects on GitHub

“…Another study oriented towards Android applications was executed by Lin et al, where more than 3.5 million GitHub repositories were analyzed, and they identified more than 12,000 non-trivial and real-world Android apps [22]. They observed that only 8% of applications have any automated tests, only 6% unit tests and UI testing is less adopted than unit testing (4%).…”

ER catcher

“…To aid the developers with development of concurrent software, Android provisions several new concurrency constructs in the form of libraries, such as AsyncTask and Looper. Nevertheless, concurrency is a major source of confusion for developers [26] and remains among the top 5 reasons for defects in Android apps [51].…”