Similarity-based Web Element Localization for Robust Test Automation

Abstract

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This paper proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be four milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.

Robust web element identification for evolving applications by considering visual overlaps

“…Similo is another approach of multi-locators based on a weighted similarity score computed on the differences between the locator parameters of the web element to locate and each of the web elements on the current web page [16]. Unlike the multi-locator approach proposed by Leotta et al [17], which uses five single-locators that uniquely identify precisely one (or none) web element each, Similo combine comparisons on multiple attributes in a single score.…”

“…In the Similo study [16], currently available as a preprint at arXiv, the authors used 14 different locator parameters with corresponding comparison operators and weights summarized in Figure 2. The locator parameters Tag, Class, Name, Id, HRef, Alt, XPath and ID relative XPath, Location, Visible Text are collected directly from the DOM-tree.…”

“…2: Graphical representation of the computation of similarity score between two different sets of locator parameters. [16].…”

“…In this section, we report the goal, the research questions, the methodology, and the results of the empirical evaluation we performed when comparing VON Similo to the baseline. Similo was selected as the baseline since a previous study [16] (currently only available as a preprint) showed that it was able to locate more web elements when compared to the multi-locator approach proposed by Leotta et al [17].…”

“…Nass et al proposed an approach called similarity-based web element localization (Similo) [16], which calculates a weighted similarity score between the target web element in a previous version, and all web elements (candidates), in a revised version, of a web application. The target web element contains the desired properties (e.g., attributes) that are compared with each candidate.…”