Web Page Segmentation Revisited

Kiesel, Johannes; Kneist, Florian; Meyer, Lars; Komlossy, Kristof; Stein, Benno; Potthast, Martin

doi:10.1145/3340531.3412782

Cited by 13 publications

(11 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conduct a comprehensive assessment of our method including both quantitative and qualitative analyses. In the quantitative aspect, we evaluate our approach using a dataset comprising 1,969 web pages sourced from the Webis-WebSeg-20 dataset [16]. Our results demonstrate a notable performance, with our method achieving a 64% F B 3 score, surpassing VIPS and MMDetection by 19% and 10%, respectively, on the same dataset.…”

Section: Introductionmentioning

confidence: 92%

“…The optimized segmentation layout is more comprehensible and reduces redundant zones in each section, as shown in figure 7b. We conduct our evaluation on the Webis-WebSeg-20 [16] dataset, which is the largest publicly accessible set of data in terms of web segmentation. The dataset contains more than 8,000 web pages from more than 5,500 different domains.…”

Section: Visual Optimizationmentioning

confidence: 99%

“…We adopted a comprehensive framework for assessing the performance of web page segmentation algorithms [16]. This approach allows us to enhance the comparability and consistency of our method's performance with other evaluated models that have been tested on the same dataset.…”

Section: Metricsmentioning

confidence: 99%

“…In the context of clustering, defining a similarity function is essential as it allows us to quantify the distance or dissimilarity between elements within a cluster. In such a manner, Kiesel et al [16] have introduced the atomic element concept; they are (1) pixels, (2) DOM nodes, and (3) characters. The authors adapt the extended BCubed measurement from clustering, which examines the P B 3 (Equation 2), R B 3 (Equation 3), and F B 3 (Equation 4) for each data instance [16].…”

Section: Metricsmentioning

confidence: 99%

See 3 more Smart Citations

A DOM-structural Cohesion Analysis Approach for Segmentation of Modern Web Pages

Huynh,

Le,

Nguyen

et al. 2024

Preprint

View full text Add to dashboard Cite

Web page segmentation is a fundamental technique applied in information retrieval systems to enhance web crawling tasks and information extraction. Its purpose is to gain deep insights from crawling results and extract the main content of a webpage by disregarding the irrelevant regions. Over time, several solutions have been proposed to address the segmentation problem using different approaches and learning strategies. Among these, the structural cue, which is a characteristic of the DOM tree, is widely utilized as a primary factor in segmentation models. In this paper, we propose a novel technique for web page segmentation using DOM-structural cohesion analysis. Our approach involves generating blocks that represent groups of DOM subtrees with similar tag structures. By analyzing the cohesion within each generated block and comparing detailed information such as types, attributes, and visual cues of web page elements, we can effectively maintain or reconstruct the segmentation layout. Additionally, we employ the Canny algorithm to optimize the segmentation result by reducing redundant spaces, resulting in a more correct segmentation. We evaluate the effectiveness of our approach using a dataset of 1,969 web pages. The approach achieves 64% on the FB3 score, surpassing existing state-of-the-art methods. The proposed DOM-structural cohesion analysis has the potential for improving web page segmentation and its various applications.

show abstract

Section: Introductionmentioning

confidence: 92%

Section: Visual Optimizationmentioning

confidence: 99%

Section: Metricsmentioning

confidence: 99%

Section: Metricsmentioning

confidence: 99%

See 2 more Smart Citations

A DOM-structural Cohesion Analysis Approach for Segmentation of Modern Web Pages

Huynh,

Le,

Nguyen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Efficiently representing real-world websites is a longstanding challenge in web understanding (Wu et al, 2023), including subtasks like web information extraction (Chang et al, 2006) and web segmentation (Kiesel et al, 2020).…”

Section: Website Representationsmentioning

confidence: 99%

UAS Navigation in the Real World Using Visual Observation

2022

View full text Add to dashboard Cite

We propose the problem of conversational web navigation, where a digital agent controls a web browser and follows user instructions to solve realworld tasks in a multi-turn dialogue fashion. To support this problem, we introduce WEBLINX -a large-scale benchmark of 100K interactions across 2300 expert demonstrations of conversational web navigation. Our benchmark covers a broad range of patterns on over 150 real-world websites and can be used to train and evaluate agents in diverse scenarios. Due to the magnitude of information present, Large Language Models (LLMs) cannot process entire web pages in real-time. To solve this bottleneck, we design a retrieval-inspired model that efficiently prunes HTML pages by ranking relevant elements. We use the selected elements, along with screenshots and action history, to assess a variety of models for their ability to replicate human behavior when navigating the web. Our experiments span from small text-only to proprietary multimodal LLMs. We find that smaller finetuned decoders surpass the best zero-shot LLMs (including GPT-4V), but also larger finetuned multimodal models which were explicitly pretrained on screenshots. However, all finetuned models struggle to generalize to unseen websites. Our findings highlight the need for large multimodal models that can generalize to novel settings. Our code, data and models are available for research: https://mcgillnlp.github.io/weblinx.

show abstract