TIE: Topological Information Enhanced Structural Reading Comprehension on Web Pages

Abstract

Recently, the structural reading comprehension (SRC) task on web pages has attracted increasing research interests. Although previous SRC work has leveraged extra information such as HTML tags or XPaths, the informative topology of web pages is not effectively exploited. In this work, we propose a Topological Information Enhanced model (TIE), which transforms the token-level task into a tag-level task by introducing a two-stage process (i.e. node locating and answer refining). Based on that, TIE integrates Graph Attention Network (GAT) and Pre-trained Language Model (PLM) to leverage the topological information of both logical structures and spatial structures. Experimental results demonstrate that our model outperforms strong baselines and achieves state-ofthe-art performances on the web-based SRC benchmark WebSRC at the time of writing.

Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding

“…Tanaka et al, as well as Chen et al, introduced web-oriented reading comprehension datasets VisualMRC [35] and WebSRC [4], respectively, requiring models to understand the spatial structure of webpages as well as the textual content to answer corresponding questions. At the same time, many approaches [27,32,39] employ graph neural networks to encode node relationships in webpages. Additionally, large language models [13] have been proven to possess strong webpage understanding capabilities via few-shot learning.…”