An adaptive annotation approach for biomedical entity and relation recognition

Abstract

In this article, we demonstrate the impact of interactive machine learning: we develop biomedical entity recognition dataset using a human-into-the-loop approach. In contrary to classical machine learning, human-in-the-loop approaches do not operate on predefined training or test sets, but assume that human input regarding system improvement is supplied iteratively. Here, during annotation, a machine learning model is built on previous annotations and used to propose labels for subsequent annotation. To demonstrate that such interactive and iterative annotation speeds up the development of quality dataset annotation, we conduct three experiments. In the first experiment, we carry out an iterative annotation experimental simulation and show that only a handful of medical abstracts need to be annotated to produce suggestions that increase annotation speed. In the second experiment, clinical doctors have conducted a case study in annotating medical terms documents relevant for their research. The third experiment explores the annotation of semantic relations with relation instance learning across documents. The experiments validate our method qualitatively and quantitatively, and give rise to a more personalized, responsive information extraction technology.

new/s/leak – Information Extraction and Visualization for Investigative Data Journalists

“…Additional features for manual data curation will enhance data quality for analysis while ensuring protection of sources and compliance with legal issues. We will also integrate adaptive annotation machine learning approach (Yimam et al, 2016) into new/s/leak to automatically identify interesting objects based on the journalists' interaction and feedback. Further, we will investigate pulling in other information from linked open data and the web.…”

Semantic Annotation of Medical Documents in CDA Context

“…The extraction of information from textual medical documents is a very active field of research. Different studies take into account the task of processing non standardized medical data considering text mining and statistical methods in order to identify medical concepts [9], also exploiting a human feedback [20].…”

Cybersecurity Named Entity Recognition Using Multi-Modal Ensemble Learning

“…from unstructured texts [5]- [7]. In recent years, many named entity recognition models have been proposed to help users to find objects of value information, including recommendation system [8], [9], question answering [10], [11] and biomedical [12], [13]. In the domain of cybersecurity, security information extraction have attracted many research efforts from different perspectives.…”