Automated reporting of GUI design violations for mobile apps

Fan

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

et al. 2019

Mobile apps are now ubiquitous. Before developing a new app, the development team usually endeavors painstaking efforts to review many existing apps with similar purposes. The review process is crucial in the sense that it reduces market risks and provides inspiration for app development. However, manual exploration of hundreds of existing apps by different roles (e.g., product manager, UI/UX designer, developer) in a development team can be ineffective. For example, it is difficult to completely explore all the functionalities of the app in a short period of time. Inspired by the conception of storyboard in movie production, we propose a system, StoryDroid, to automatically generate the storyboard for Android apps, and assist different roles to review apps efficiently. Specifically, StoryDroid extracts the activity transition graph and leverages static analysis techniques to render UI pages to visualize the storyboard with the rendered pages. The mapping relations between UI pages and the corresponding implementation code (e.g., layout code, activity code, and method hierarchy) are also provided to users. Our comprehensive experiments unveil that StoryDroid is effective and indeed useful to assist app development. The outputs of StoryDroid enable several potential applications, such as the recommendation of UI design and layout code.

Section: Discussionmentioning

confidence: 99%

StoryDroid: Automated Generation of Storyboard for Android Apps

Fan

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

et al. 2019

“…Similarly, PUMA [61] is a framework that includes a generic UI automator that provides a random exploration of mobile apps using the basic monkey exploration strategy. GVT [62] is an automated approach for verifying the UI of a mobile app against the intended design specifications.…”

Section: A Dynamic Approachesmentioning

confidence: 99%

AMOGA: A Static-Dynamic Model Generation Strategy for Mobile Apps Testing

et al. 2019

In the past few years, mobile devices have been increasingly replacing traditional computers as their capabilities such as CPU computation, memory, RAM size, and many more, are being enhanced almost to the level of conventional computers. These capabilities are being exploited by mobile apps developers to produce apps that offer more functionalities and optimized performance. To ensure acceptable quality and to meet their specifications (e.g., design), mobile apps need to be tested thoroughly. As the testing process is often tedious, test automation can be the key to alleviating such laborious activities. In the context of the Android-based mobile apps, researchers and practitioners have proposed many approaches to automate the testing process mainly on the creation of the test suite. Although useful, most existing approaches rely on reverse engineering a model of the application under test for test case creation. Often, such approaches exhibit a lack of comprehensiveness as the application model does not capture the dynamic behavior of the applications extensively due to the incompleteness of reverse engineering approaches. To address this issue, this paper proposes AMOGA, a strategy that uses a hybrid, static-dynamic approach for generating user interface model from mobile apps for modelbased testing. AMOGA implements a novel crawling technique that uses the event list of UI element associated with each event to dynamically exercise the events ordering at the run-time to explore the applications' behavior. An experimental evaluation was performed to assess the effectiveness of our strategy by measuring the code coverage and the fault detection capability through the use of mutation testing concept. Results of the experimental assessment showed that AMOGA represents an alternative approach for model-based testing of mobile apps by generating comprehensive models to improve the coverage of the applications. The strategy proved its effectiveness by achieving high code coverage and mutation score for different applications.

Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

“…To do this, we look to past work on detecting design violations in mobile apps [30]. A design violation in the context of mobile apps has been defined as a mismatch between the attribute vectors of two GUI-components that exist both in a mobile GUI mockup and implementation, where the attribute vectors can be represented as a a four-tuple in the form (<x-position,y-position>, <height,width>, <text>, <image>) [30]. In this work the authors performed a grounded-theory survey on an industrial dataset of design violations and derived a taxonomy.…”

Section: Evolutionary Gui Changesmentioning

confidence: 99%

“…After corresponding pairs of leaf node components have been identified, Gcat must then detect GUI changes between screens. Gcat first employs Perceptual Image Differencing (PID), an image differencing algorithm modeled after the human visual system that has been successfully applied in past research on detecting GUI differences [28][29][30]. PID helps to identify a set of potential changes based on visual differences between images.…”

Section: Gui-change Resolutionmentioning

confidence: 99%

Detecting and summarizing GUI changes in evolving mobile apps

Moran

Watson

Hoskins

et al. 2018

Self Cite

Mobile applications have become a popular software development domain in recent years due in part to a large user base, capable hardware, and accessible platforms. However, mobile developers also face unique challenges, including pressure for frequent releases to keep pace with rapid platform evolution, hardware iteration, and user feedback. Due to this rapid pace of evolution, developers need automated support for documenting the changes made to their apps in order to aid in program comprehension. One of the more challenging types of changes to document in mobile apps are those made to the graphical user interface (GUI) due to its abstract, pixel-based representation. In this paper, we present a fully automated approach, called Gcat, for detecting and summarizing GUI changes during the evolution of mobile apps. Gcat leverages computer vision techniques and natural language generation to accurately and concisely summarize changes made to the GUI of a mobile app between successive commits or releases. We evaluate the performance of our approach in terms of its precision and recall in detecting GUI changes compared to developer specified changes, and investigate the utility of the generated change reports in a controlled user study. Our results indicate that Gcat is capable of accurately detecting and classifying GUI changes -outperforming developers -while providing useful documentation. CCS CONCEPTS• Software and its engineering → Software development process management; Software development methods;