Web Element Identification by Combining NLP and Heuristic Search for Web Testing

Kirinuki, Hiroyuki; Matsumoto, Shinsuke; Higo, Yoshiki; Kusumoto, Shoichi

doi:10.1109/saner53432.2022.00123

Cited by 2 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A study by Kirinuki et al attempts to solve the locator maintenance problem by not relying on attributes and the structure of the DOM and instead leverages NLP with heuristic search to identify web elements in web pages from natural-languagelike test cases [22]. An example of such a test step could be: enter 'admin' in 'username'.…”

Section: B Resilient Locatorsmentioning

confidence: 99%

“…One such example is SocraTest, a vision of a framework for conversational testing agents that could aid a human software tester by performing tasks autonomously [21]. Recent studies utilize natural language processing (NLP) with heuristic search and the DOM structure to identify web elements in web applications [22] or use LLMs to generate text inputs for GUI applications based on semantic understanding and GUI application context [23], [24], [25]. The proposed solution in this paper is based on the hypothesis that we can improve web element localization even further by combining an LLM with a traditional algorithm to take advantage of some of the benefits of the LLM, e.g., its assumed semantic understanding and contextual awareness, while utilizing the speed of the conventional algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Similarity-based Web Element Localization for Robust Test Automation

Nass

Alégroth

Feldt

et al. 2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This paper proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be four milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.

show abstract

Section: B Resilient Locatorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Similarity-based Web Element Localization for Robust Test Automation

Nass

Alégroth

Feldt

et al. 2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

Improving Web Element Localization by Using a Large Language Model

Nass,

Alégroth,

Feldt

2024

Software Testing Verif & Rel

View full text Add to dashboard Cite

Web‐based test automation heavily relies on accurately finding web elements. Traditional methods compare attributes but do not grasp the context and meaning of elements and words. The emergence of large language models (LLMs) like GPT‐4, which can show human‐like reasoning abilities on some tasks, offers new opportunities for software engineering and web element localization. This paper introduces and evaluates VON Similo LLM, an enhanced web element localization approach. Using an LLM, it selects the most likely web element from the top‐ranked ones identified by the existing VON Similo method, ideally aiming to get closer to human‐like selection accuracy. An experimental study was conducted using 804 web element pairs from 48 real‐world web applications. We measured the number of correctly identified elements as well as the execution times, comparing the effectiveness and efficiency of VON Similo LLM against the baseline algorithm. In addition, motivations from the LLM were recorded and analysed for 140 instances. VON Similo LLM demonstrated improved performance, reducing failed localizations from 70 to 40 (out of 804), a 43% reduction. Despite its slower execution time and additional costs of using the GPT‐4 model, the LLM's human‐like reasoning showed promise in enhancing web element localization. LLM technology can enhance web element localization in GUI test automation, reducing false positives and potentially lowering maintenance costs. However, further research is necessary to fully understand LLMs' capabilities, limitations and practical use in GUI testing.

show abstract

Web Element Identification by Combining NLP and Heuristic Search for Web Testing

Cited by 2 publications

References 21 publications

Similarity-based Web Element Localization for Robust Test Automation

Similarity-based Web Element Localization for Robust Test Automation

Improving Web Element Localization by Using a Large Language Model

Contact Info

Product

Resources

About