MyMiner: a web application for computer-assisted biocuration and text annotation

Salgado, David; Krallinger, Martin; Depaule, Marc; Drula, Élodie; Tendulkar, Ashish V.; Leitner, Florian; Valencia, Alfonso; Marcelle, Christophe

doi:10.1093/bioinformatics/bts435

Cited by 46 publications

(29 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many applications for different domains have been built in order to semi-automatically annotate text as the user is working, updating future files with machine learning output based on previous annotations 13–18. These efforts all seek to decrease annotation time, but in our study we focus on the role of a single pre-annotation set for particular named entities in the clinical domain.…”

Section: Background and Significancementioning

confidence: 99%

Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements

Lingren

Deléger

Molnar

et al. 2014

J Am Med Inform Assoc

View full text Add to dashboard Cite

ObjectiveTo present a series of experiments: (1) to evaluate the impact of pre-annotation on the speed of manual annotation of clinical trial announcements; and (2) to test for potential bias, if pre-annotation is utilized.MethodsTo build the gold standard, 1400 clinical trial announcements from the clinicaltrials.gov website were randomly selected and double annotated for diagnoses, signs, symptoms, Unified Medical Language System (UMLS) Concept Unique Identifiers, and SNOMED CT codes. We used two dictionary-based methods to pre-annotate the text. We evaluated the annotation time and potential bias through F-measures and ANOVA tests and implemented Bonferroni correction.ResultsTime savings ranged from 13.85% to 21.5% per entity. Inter-annotator agreement (IAA) ranged from 93.4% to 95.5%. There was no statistically significant difference for IAA and annotator performance in pre-annotations.ConclusionsOn every experiment pair, the annotator with the pre-annotated text needed less time to annotate than the annotator with non-labeled text. The time savings were statistically significant. Moreover, the pre-annotation did not reduce the IAA or annotator performance. Dictionary-based pre-annotation is a feasible and practical method to reduce the cost of annotation of clinical named entity recognition in the eligibility sections of clinical trial announcements without introducing bias in the annotation process.

show abstract

Section: Background and Significancementioning

confidence: 99%

Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements

Lingren

Deléger

Molnar

et al. 2014

J Am Med Inform Assoc

View full text Add to dashboard Cite

show abstract

“…Both used dictionary lookup, but in different ways. The first system [34] used Lucene [35] to maintain documents in the test set and a set of searches was performed (one for each method term). The top 100 documents for each search were recovered, and a method identifier was associated to a document if the score during the search was above certain threshold.…”

Section: Dictionary Look-upmentioning

confidence: 99%

Towards a Protein–Protein Interaction information extraction system: Recognizing named entities

Dánger

Pla

Molina

et al. 2014

Knowledge-Based Systems

View full text Add to dashboard Cite

The majority of biological functions of any living being are related to ProteinProtein Interactions (PPI). PPI discoveries are reported in form of research publications whose volume grows day after day. Consequently, automatic PPI information extraction systems are a pressing need for biologists. In this paper we are mainly concerned with the named entity detection module of PPIES (the PPI Information extraction system we are implementing) which recognizes twelve entity types relevant in PPI context. It is composed of two sub-modules: a dictionary look-up with extensive normalization and acronym detection, and a Conditional Random Field classifier. The dictionary look-up module has been tested with Interaction Method Task (IMT), and it improves by approximately 10% the current solutions that do not use Machine Learning (ML). The second module has been used to create a classifier using the Joint Workshop on Natural Language Processing in Biomedicine and its Applications (JNLPBA'04) data set. It does not use any external resources, or complex or ad-hoc post-processing, and obtains 77.25%, 75.04% and 76.13 for precision, recall, and F1-measure, respectively, improving all previous results obtained for this data set.

show abstract

“…MyMiner [76] is a web-based tool supporting a range of tasks in text curation, such as document labeling, annotation of entities and binary relationships and entity normalization. It participated in the BioCreative III IAT [77] and has been used for curation of PPIs [78] and for assisting manual document classification in the BioCreative III ACT [79].…”

Section: Mmax2 Eml Research (Germany)mentioning

confidence: 99%

A survey on annotation tools for the biomedical literature

Neves¹,

Leser²

2012

Briefings in Bioinformatics

View full text Add to dashboard Cite

New approaches to biomedical text mining crucially depend on the existence of comprehensive annotated corpora. Such corpora, commonly called gold standards, are important for learning patterns or models during the training phase, for evaluating and comparing the performance of algorithms and also for better understanding the information sought for by means of examples. Gold standards depend on human understanding and manual annotation of natural language text. This process is very time-consuming and expensive because it requires high intellectual effort from domain experts. Accordingly, the lack of gold standards is considered as one of the main bottlenecks for developing novel text mining methods. This situation led the development of tools that support humans in annotating texts. Such tools should be intuitive to use, should support a range of different input formats, should include visualization of annotated texts and should generate an easy-to-parse output format. Today, a range of tools which implement some of these functionalities are available. In this survey, we present a comprehensive survey of tools for supporting annotation of biomedical texts. Altogether, we considered almost 30 tools, 13 of which were selected for an in-depth comparison. The comparison was performed using predefined criteria and was accompanied by hands-on experiences whenever possible. Our survey shows that current tools can support many of the tasks in biomedical text annotation in a satisfying manner, but also that no tool can be considered as a true comprehensive solution.

show abstract

MyMiner: a web application for computer-assisted biocuration and text annotation

Cited by 46 publications

References 9 publications

Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements

Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements

Towards a Protein–Protein Interaction information extraction system: Recognizing named entities

A survey on annotation tools for the biomedical literature

Contact Info

Product

Resources

About