Structured Literature Image Finder: Parsing Text and Figures in Biomedical Literature

Ahmed, Amr; Arnold, Andrew O.; Coelho, Luís Pedro; Kangas, Joshua; Sheikh, Abdul-Saboor; Xing, Eric P.; Cohen, William W.; Murphy, Robert F.

doi:10.2139/ssrn.3199484

Cited by 4 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[22]. A decade later, Ahmed et al [3,4] improved the model for mining captioned figures. The latest version combines text-mining and image processing to extract structured information from biomedical literature.…”

Section: Related Workmentioning

confidence: 99%

“…For each category, we randomly reserved 25% of the images for a testing set and trained the SVM model on the remaining 75%. We used sklearn's grid search method to tune the SVM parameters (kernel, gamma, and penalty parameter) 3 . Once the model parameters are tuned, we evaluate the model by using the testing set and then trained the final model with all images.…”

Section: Figure Classificationmentioning

confidence: 99%

See 1 more Smart Citation

Viziometrics: Analyzing Visual Information in the Scientific Literature

Lee

West

Howe

2018

IEEE Trans. Big Data

View full text Add to dashboard Cite

Scientific results are communicated visually in the literature through diagrams, visualizations, and photographs. These information-dense objects have been largely ignored in bibliometrics and scientometrics studies when compared to citations and text. In this paper, we use techniques from computer vision and machine learning to classify more than 8 million figures from PubMed into 5 figure types and study the resulting patterns of visual information as they relate to impact. We find that the distribution of figures and figure types in the literature has remained relatively constant over time, but can vary widely across field and topic. Remarkably, we find a significant correlation between scientific impact and the use of visual information, where higher impact papers tend to include more diagrams, and to a lesser extent more plots and photographs. To explore these results and other ways of extracting this visual information, we have built a visual browser to illustrate the concept and explore design alternatives for supporting viziometric analysis and organizing visual information. We use these results to articulate a new research agenda -viziometrics -to study the organization and presentation of visual information in the scientific literature.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Figure Classificationmentioning

confidence: 99%

Viziometrics: Analyzing Visual Information in the Scientific Literature

Lee

West

Howe

2018

IEEE Trans. Big Data

View full text Add to dashboard Cite

show abstract

“…An existing approach towards this problem involves finding the entities from texts (or even a search box) and associating them with images from medical literature (using caption processing, image processing, and topic discovery), and can be found in the Structured Literature Image Finder [1]. The training data used manually annotated images from different subcellular locations.…”

Section: The Role Of Visual Rhetoric In Semantic Multimedia and Relatmentioning

confidence: 99%

The Role of Visual Rhetoric in Semantic Multimedia: Strategies for Decision Making in Times of Crisis

Brașoveanu

Dzițac

2014

INT J COMPUT COMMUN

View full text Add to dashboard Cite

As semantic multimedia is approaching mainstream, even the great improvements that can be seen in its classic schools, like the data mining inspired Information Retrieval based on metadata analysis, or Computer Vision, might not be enough. We identify a new group that gains traction in the semantic multimedia community and which uses as starting point developments from psychology and visual communication. For the purposes of this article we restrict our domain to visual rhetoric as we consider it to yield the biggest potential for future developments. Living in times when the periods between crises seem to be shorter and shorter, we look at how developments in semantic multimedia can be used for predicting and overcoming crises. We analyze at least 2 aspects related to this: using information visualization to understand the evolution of crises and creating multi-layered semantic multimedia technologies that can easily be adapted to use a variety of sources and solve problems from different domains. In both cases we show how techniques inspired by visual rhetoric (information linking, framing, composition) in conjunction with named entity recognition offer a lot of benefits. The section related to multi-layered semantic multimedia technologies also draws on the lessons learned while designing a prototype application aimed at improving tourism decision making process. The article ends with a discussion on evaluation methods for multi-layered semantic technologies applications. We look at how to evaluate them on both levels: mechanisms (information linking versus raw named entity recognition when generating visuals, for example), and decision making strategies (Do such systems actually solve real problems related to crises, create jobs or at least can they be repurposed to solve other problems than the one with which we have started?).

show abstract

“…The structured literature image finder (SLIF) system applies an existing optical character recognition (OCR) system to recognize figure text and identify potential image pointers. SLIF then parses text and figures in biomedical literature by matching image pointers in images and captions [ 7 ]. Other researchers have also applied existing OCR tools to extract figure text and then incorporate the figure text for applications, e.g., image and document retrieval [ 5 , 11 ].…”

Section: Introductionmentioning

confidence: 99%

DeTEXT: A Database for Evaluating Text Extraction from Biomedical Literature Figures

et al. 2015

View full text Add to dashboard Cite

Hundreds of millions of figures are available in biomedical literature, representing important biomedical experimental evidence. Since text is a rich source of information in figures, automatically extracting such text may assist in the task of mining figure information. A high-quality ground truth standard can greatly facilitate the development of an automated system. This article describes DeTEXT: A database for evaluating text extraction from biomedical literature figures. It is the first publicly available, human-annotated, high quality, and large-scale figure-text dataset with 288 full-text articles, 500 biomedical figures, and 9308 text regions. This article describes how figures were selected from open-access full-text biomedical articles and how annotation guidelines and annotation tools were developed. We also discuss the inter-annotator agreement and the reliability of the annotations. We summarize the statistics of the DeTEXT data and make available evaluation protocols for DeTEXT. Finally we lay out challenges we observed in the automated detection and recognition of figure text and discuss research directions in this area. DeTEXT is publicly available for downloading at http://prir.ustb.edu.cn/DeTEXT/.

show abstract

Structured Literature Image Finder: Parsing Text and Figures in Biomedical Literature

Cited by 4 publications

References 9 publications

Viziometrics: Analyzing Visual Information in the Scientific Literature

Viziometrics: Analyzing Visual Information in the Scientific Literature

The Role of Visual Rhetoric in Semantic Multimedia: Strategies for Decision Making in Times of Crisis

DeTEXT: A Database for Evaluating Text Extraction from Biomedical Literature Figures

Contact Info

Product

Resources

About