Benchmarking infrastructure for mutation text mining

Klein, Artjom; Riazanov, Alexandre; Hindle, Matthew; Baker, Christopher J. O.

doi:10.1186/2041-1480-5-11

Cited by 5 publications

(12 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Micro and macro scores were reported by, 30 , 41 whereas 26 , 42 reported micro across documents, and macro across the classes. Micro scores were used by 41 for class-level results.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Data extraction methods for systematic review (semi)automation: A living systematic review

et al. 2021

View full text Add to dashboard Cite

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search MEDLINE, Institute of Electrical and Electronics Engineers (IEEE), arXiv, and the dblp computer science bibliography databases. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This iteration of the living review includes publications up to a cut-off date of 22 April 2020. Results: In total, 53 publications are included in this version of our review. Of these, 41 (77%) of the publications addressed extraction of data from abstracts, while 14 (26%) used full texts. A total of 48 (90%) publications developed and evaluated classifiers that used randomised controlled trials as the main target texts. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. A description of their datasets was provided by 49 publications (94%), but only seven (13%) made the data publicly available. Code was made available by 10 (19%) publications, and five (9%) implemented publicly available tools. Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of systematic review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. The lack of publicly available gold-standard data for evaluation, and lack of application thereof, makes it difficult to draw conclusions on which is the best-performing system for each data extraction target. With this living review we aim to review the literature continually.

show abstract

“…Micro and macro scores were reported by, 30 , 41 whereas 26 , 42 reported micro across documents, and macro across the classes. Micro scores were used by 41 for class-level results.…”

Section: Resultsmentioning

confidence: 99%

“… 49 Five publications focused on extracting data specifically from epidemiology research, or included text from cohort studies as well as RCT text. 26 , 42 , 49 – 51 More publications mining data from surveys or case series might have been found if our search and review had concentrated on these types of texts.…”

Section: Resultsmentioning

confidence: 99%

Data extraction methods for systematic review (semi)automation: A living systematic review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In addition, the Disease Ontology[83], while intended for the broader study of disease, annotates terms with OMIM identifiers, and can thus be utilized for the analysis of associated variants. Finally, the Mutation Impact Extraction Ontology (MIEO)[84] is an open-source tool which assists in documenting how variants and their associated properties and phenotypic effects are described in natural language, which can be used to facilitate the automated extraction of variants from the literature.…”

Section: Variant Databases and Resourcesmentioning

confidence: 99%

Towards Precision Medicine: Advances in Computational Approaches for the Analysis of Human Variants

Peterson

Doughty

Kann

2013

Journal of Molecular Biology

121

122

View full text Add to dashboard Cite

Variations and similarities in our individual genomes are part of our history, our heritage, and our identity. Some human genomic variants are associated with common traits such as hair and eye color, while others are associated with susceptibility to disease or response to drug treatment. Identifying the human variations producing clinically relevant phenotypic changes is critical for providing accurate and personalized diagnosis, prognosis, and treatment for diseases. Furthermore, a better understanding of the molecular underpinning of disease can lead to development of new drug targets for precision medicine. Several resources have been designed for collecting and storing human genomic variations in highly structured, easily accessible databases. Unfortunately, a vast amount of information about these genetic variants and their functional and phenotypic associations is currently buried in the literature, only accessible by manual curation or sophisticated text mining technology to extract the relevant information. In addition, the low cost of sequencing technologies coupled with increasing computational power has enabled the development of numerous computational methodologies to predict the pathogenicity of human variants. This review provides a detailed comparison of current human variant resources, including HGMD, OMIM, ClinVar, and UniProt/Swiss-Prot, followed by an overview of the computational methods and techniques used to leverage the available data to predict novel deleterious variants. We expect these resources and tools to become the foundation for understanding the molecular details of genomic variants leading to disease, which in turn will enable the promise of precision medicine.

show abstract

“…Much of the work describing the efforts to mine literature on individual amino acids is reviewed by [ 8 ], and the reader is referred to that work for detailed background. Klein et al [ 9 ] proposed an infrastructure for evaluation of programs that identify mutations. Much of the previous work involves identifying mutations.…”

Section: Introductionmentioning

confidence: 99%

ResidueFinder: extracting individual residue mentions from protein literature

Becker

Jakobsson

2021

J Biomed Semant

View full text Add to dashboard Cite

Background The revolution in molecular biology has shown how protein function and structure are based on specific sequences of amino acids. Thus, an important feature in many papers is the mention of the significance of individual amino acids in the context of the entire sequence of the protein. MutationFinder is a widely used program for finding mentions of specific mutations in texts. We report on augmenting the positive attributes of MutationFinder with a more inclusive regular expression list to create ResidueFinder, which finds mentions of native amino acids as well as mutations. We also consider parameter options for both ResidueFinder and MutationFinder to explore trade-offs between precision, recall, and computational efficiency. We test our methods and software in full text as well as abstracts. Results We find there is much more variety of formats for mentioning residues in the entire text of papers than in abstracts alone. Failure to take these multiple formats into account results in many false negatives in the program. Since MutationFinder, like several other programs, was primarily tested on abstracts, we found it necessary to build an expanded regular expression list to achieve acceptable recall in full text searches. We also discovered a number of artifacts arising from PDF to text conversion, which we wrote elements in the regular expression library to address. Taking into account those factors resulted in high recall on randomly selected primary research articles. We also developed a streamlined regular expression (called “cut”) which enables a several hundredfold speedup in both MutationFinder and ResidueFinder with only a modest compromise of recall. All regular expressions were tested using expanded F-measure statistics, i.e., we compute Fβ for various values of where the larger the value of β the more recall is weighted, the smaller the value of β the more precision is weighted. Conclusions ResidueFinder is a simple, effective, and efficient program for finding individual residue mentions in primary literature starting with text files, implemented in Python, and available in SourceForge.net. The most computationally efficient versions of ResidueFinder could enable creation and maintenance of a database of residue mentions encompassing all articles in PubMed.

show abstract

Benchmarking infrastructure for mutation text mining

Cited by 5 publications

References 41 publications

Data extraction methods for systematic review (semi)automation: A living systematic review

Data extraction methods for systematic review (semi)automation: A living systematic review

Towards Precision Medicine: Advances in Computational Approaches for the Analysis of Human Variants

ResidueFinder: extracting individual residue mentions from protein literature

Contact Info

Product

Resources

About