Neural Collaborative Graph Machines for Table Structure Recognition

Abstract

Recently, Table Structure Recognition (TSR) task, aiming at identifying table structure into machine readable formats, has received increasing interest in the community. While impressive success, most single table componentbased methods can not perform well on unregularized table cases distracted by not only complicated inner structure but also exterior capture distortion. In this paper, we raise it as Complex TSR problem, where the performance degeneration of existing methods is attributable to their inefficient component usage and redundant post-processing. To mitigate it, we shift our perspective from table component extraction towards the efficient multiple components leverage, which awaits further exploration in the field. Specifically, we propose a seminal method, termed GrabTab, equipped with newly proposed Component Deliberator. Thanks to its progressive deliberation mechanism, our GrabTab can flexibly accommodate to most complex tables with reasonable components selected but without complicated postprocessing involved. Quantitative experimental results on public benchmarks demonstrate that our method significantly outperforms the state-of-the-arts, especially under more challenging scenes.

Grab What You Need: Rethinking Complex Table Structure Recognition with Flexible Components Deliberation

“…Among, the first three columns indicate the performance on recognizing complex tables containing more severe distractors. Compared with existing methods, the F1-score of our GrabTab can beat the second best method, NCGM (Liu et al 2022), by 4.5% on SciTSR-COMP-A dataset, while the apparent performance improvement is also witnessed on WTW and SciTSR-COMP datasets. This phenomenon further confirms that simply focusing on single component extraction is not the optimal solution.…”

“…As aforementioned, our GrabTab extracts table elements and their relations as candidate components, which is expected to provide useful information for the chief separator component. To achieve this goal, we inherit the relation extraction method from a off-the-shell work, NCGM (Liu et al 2022). Specifically, for N table elements, the "collaborative graph embeddings" output by NCGM is employed as element tokens in our GrabTab: E ele = {e 1 , e 2 , ..., e N } ∈ R N ×de .…”

“…We evaluate our method on the following benchmark datasets under both complex and regularized table scenarios. ICDAR-2013 (Göbel et al 2013), ICDAR-2019 (Gao et al 2019), WTW (Long et al 2021), UNLV (Shahab et al 2010), SciTSR (Chi et al 2019), SciTSR-COMP (Chi et al 2019) and SciTSR-COMP-A (Liu et al 2022) are evaluated under protocol of physical structure recognition, while TableBank (Li et al 2020) and PubTabNet (Zhong, ShafieiBavani, and Jimeno Yepes 2020) are adopted to evaluate the logical structure recognition performance. Furthermore, WTW (Long et al 2021), SciTSR-COMP (Chi et al 2019) and SciTSR-COMP-A (Liu et al 2022) are employed as complex TSR datasets with more challenging distractors involved, while the rest are the regularized ones.…”

“…This phenomenon further confirms that simply focusing on single component extraction is not the optimal solution. By equipped with deliberation mechanism, our GrabTab finds the silver linings behind the (Long et al 2021), FLAG-Net (Liu et al 2021), TSRFormer (w/ DQ-DETR) (Wang et al 2023), TableFormer (Nassar et al 2022), RobusTabNet (Ma et al 2023), NCGM (Liu et al 2022) and…”

“…Nevertheless, it is infeasible to implement as either component in the aforementioned methods is strongly coupled with corresponding post-processing rules, which could be mutually exclusive. For the Complex TSR, a few recent researches (Long et al 2021;Liu et al 2022;Wang et al 2023) have made attempts, whereas they only take further steps on more robust components (relationships or boundaries) extraction while the deployment of multiple components is still rarely explored. In human cognitive system (Clancey 2002;Anderson 2005;Olshausen, Anderson, and Van Essen 1993), "deliberation" is one of common behaviors when human processing daily works, such as reading or analyzing table image.…”