Automatically identifying potential regressions in the layout of responsive web pages

Abstract

SummaryProviding a good user experience on the ever‐increasing number and variety of devices being used to browse the web is a difficult, yet critical, task. With responsive web design, front‐end web developers design web pages so that they dynamically resize and rearrange content to best fit the dimensions of a device's screen. However, when making code modifications to a responsive page, developers can easily introduce regressions from the correct layout that have detrimental effects at unpredictable screen sizes. For instance, the source code change that a developer makes to improve the layout at one screen size may obscure a page's content at other sizes. Current approaches to testing are often insufficient because they rely on limited tools and error‐prone manual inspections of web pages. As such, many unintended regressions in web page layout often go undetected and ultimately manifest in production websites. To address the challenge of detecting regressions in responsive web pages, this paper presents an automated approach that extracts the responsive layout of two versions of a page and compares them, alerting developers to the differences in layout that they may wish to investigate further. We implemented the approach and empirically evaluated it on 15 real‐world responsive web pages. Leveraging code mutations that a tool automatically injected into the pages as a systematic simulation of developer changes, the experiments show that the approach was highly effective. When compared with manual and automated baseline testing techniques, it detected 12.5% and 18.75% more injected changes, respectively. Along with identifying the best parameters for the method that extracts the responsive layout, the experiments show that the approach surpasses the baselines across changes that vary in their impact, but works particularly well for subtle, hard‐to‐detect mutants, showing the benefits of automatically identifying regressions in web page layout.

Methods for Automatic Web Page Layout Testing and Analysis: A Review

“…This testing verifies whether modifying or creating new functionality introduces errors into the system. There are papers dealing with the regression testing of user interfaces [51], [52]. In [51], the authors offer a method for automatically determining reusability and repairing user interface tests.…”

Automated visual classification of DOM‐based presentation failure reports for responsive web pages

“…The first part of the algorithm involves determining the AOCs (lines 2-24) and matching them up (lines 25-28) into sets across the three viewports involved. The comparison of these AOCs takes place in a loop over each set (lines [29][30][31][32][33][34][35][36][37][38]. The algorithm obtains the image of each AOC, in the matched set-taken from a snapshot of the page at the appropriate viewport-and computes its colour histogram, a histogram of the number of pixels with a certain colour in the image (lines 31 and 33).…”

“…The ReDECHECK tool has two 'modes' [13]. In 'regression checking' mode, the tool targets regression issues by comparing the responsive layout of two versions of a web page [21,35]. In 'common failure checking' mode, ReDECHECK automatically detects the series of common responsive layout failures studied in this paper [6].…”

Effective automated repair of internationalization presentation failures in web applications using style similarity clustering and search‐based techniques

“…Several techniques [31‐37] are designed to detect and Cross‐Platform Incompatibilities and Cross‐Browser Issues in web pages by analysing HTML, CSS and Javascript. Yet other techniques, such as ReDeCheck [38‐42], WebSee [11‐13,43‐45], VFDetector [46], CANVASURE [47], Ply [48], Fighting Layout Bugs [49], Sikuli [50], and techniques based on computer vision [51] and CSS/Javascript analysis [52], detect certain types of presentation failures in web pages. There also exists a group of parallel techniques [53‐55] focusing on the detection and reporting of GUI violations in mobile apps.…”