A Two-Stage Framework for Compound Figure Separation

Abstract

Scientific literature contains large volumes of complex, unstructured figures that are compound in nature (i.e. composed of multiple images, graphs, and drawings). Separation of these compound figures is critical for information retrieval from these figures. In this paper, we propose a new strategy for compound figure separation, which decomposes the compound figures into constituent subfigures while preserving the association between the subfigures and their respective caption components. We propose a two-stage framework to address the proposed compound figure separation problem. In the first stage, the subfigure label detection module detects all subfigure labels. Then, in the subfigure detection module, the detected subfigure labels help to detect the subfigures by optimizing the feature selection process and providing the global layout information as extra features. Extensive experiments are conducted to validate the effectiveness and superiority of the proposed framework, which improves the precision by 9% compared to the benchmark.

Plot2Spectra: an automatic spectra extraction tool

“…We collected a large number of graph images from literature using the EXSCLAIM! pipeline 50,51 with the keyword “Raman” and “XANES”. Then we randomly selected 1800 images for the axis alignment task, with 800 images for training and 1000 images for validation.…”

Compound Figure Separation of Biomedical Images with Side Loss

“…The introducing of anchor has prior knowledge to object distribution which is also closer to the compound figure situation. YOLOv4 was used by Jiang et al (2021) to achieve a superior detection performance. They combined a traditional vision method with high performance of deep learning networks by detecting the sub-figure label and then optimizing the feature selection process in the sub-figure detection.…”

“…Various compound figure separation approaches have been developed ( Davila et al, 2020 ; Lee and Howe, 2015a ; Apostolova et al, 2013 ; Tsutsui and Crandall, 2017 ; Shi et al, 2019 ; Jiang et al, 2021 ; Huang et al, 2005 ), especially with recent advances in deep learning. However, previous approaches typically required resource extensive bounding box annotation to form the problem as a training detection task.…”