Automatic separation of compound figures in scientific articles

Abstract

Content-based analysis and retrieval of digital images found in scientific articles is often hindered by images consisting of multiple subfigures (compound figures). We address this problem by proposing a method to automatically classify and separate compound figures, which consists of two main steps: (i) a supervised compound figure classifier (CFC) discriminates between compound and non-compound figures using task-specific image features; and (ii) an image processing algorithm is applied to predicted compound images to perform compound figure separation (CFS). Our CFC approach is shown to achieve state-of-the-art classification performance on a published dataset. Our CFS algorithm shows superior separation accuracy on two different datasets compared to other known automatic approaches. Finally, we propose a method to evaluate the effectiveness of the CFC-CFS process chain and use it to optimize the misclassification loss of CFC for maximal effectiveness in the process chain.

Biomedical compound figure detection using deep learning and fusion techniques

“…The literature review covers existing traditional models for CFD that make use of hand-crafted features or representations [5][6][7][8][9]. They are important as benchmarks or points of comparison for the proposed deep learning model.…”

“…Taschwer and Marques [9] describe a chained CFD-CFS approach where the CFD approach in isolation used an SVM classifier with hand-crafted visual features to obtain a best run accuracy of 76.9% on a combined dataset containing among others the ImageCLEF 2015 CFD and multi-label classification (MC) subtask datasets. They used three types of visual features.…”

“…Table 7 compares the performance of the proposed model with other models. Zare and Müller [8] as well as Taschwer and Marques [9] are not directly comparable due to their different test datasets.…”

A Data Driven Approach for Compound Figure Separation Using Convolutional Neural Networks

“…We evaluate accuracy using the same metrics defined by the ImageCLEF task [8]. We briefly summarize the metrics here; please see [10], [31] for full details. For each compound figure, an accuracy ranging from 0 to 1 is defined as the number of correctly detected subfigures over the maximum of the number of ground-truth subfigures and the number of detected subfigures.…”

“…Many existing techniques first classify figures into compound or simple, and then run the compound figure separation algorithm only on the compound figures, in part because the separation algorithm is relatively slow [1], [10]. For example, separation is reported to take 0.3 seconds per compound figure in Taschwer et al [10]. This two step approach can be dangerous because if a compound figure is not recognized correctly, the subfigures can never be extracted.…”