Towards comprehensive syntactic and semantic annotations of the clinical narrative

Abstract

ObjectiveTo create annotated clinical narratives with layers of syntactic and semantic labels to facilitate advances in clinical natural language processing (NLP). To develop NLP algorithms and open source components.MethodsManual annotation of a clinical narrative corpus of 127 606 tokens following the Treebank schema for syntactic information, PropBank schema for predicate-argument structures, and the Unified Medical Language System (UMLS) schema for semantic information. NLP components were developed.ResultsThe final corpus consists of 13 091 sentences containing 1772 distinct predicate lemmas. Of the 766 newly created PropBank frames, 74 are verbs. There are 28 539 named entity (NE) annotations spread over 15 UMLS semantic groups, one UMLS semantic type, and the Person semantic category. The most frequent annotations belong to the UMLS semantic groups of Procedures (15.71%), Disorders (14.74%), Concepts and Ideas (15.10%), Anatomy (12.80%), Chemicals and Drugs (7.49%), and the UMLS semantic type of Sign or Symptom (12.46%). Inter-annotator agreement results: Treebank (0.926), PropBank (0.891–0.931), NE (0.697–0.750). The part-of-speech tagger, constituency parser, dependency parser, and semantic role labeler are built from the corpus and released open source. A significant limitation uncovered by this project is the need for the NLP community to develop a widely agreed-upon schema for the annotation of clinical concepts and their relations.ConclusionsThis project takes a foundational step towards bringing the field of clinical NLP up to par with NLP in the general domain. The corpus creation and NLP components provide a resource for research and application development that would have been previously impossible.

A French clinical corpus with comprehensive semantic annotations: development of the Medical Entity and Relation LIMSI annOtated Text corpus (MERLOT)

“…We computed our IAA values requiring an exact match between annotations, which is generally lower than a partial match. For example, Albright et al (2013) achieved an F1 measure of 0.697 in exact match, but of 0.750 in partial match. Overall, our Ogren et al (2008) for English (from 75.7 to 81.4% in entity annotation, exact match) and Oronoz et al (2015) for Spanish (from 88.63 to 90.53% in term annotation).…”

“…Notable research initiatives, in collaboration with health institutions, have annotated clinical texts: the Mayo Clinic corpus (Ogren et al 2008), the Clinical E-Science Framework (CLEF) (Roberts et al 2009), the THYME (Temporal Histories of Your Medical Events) project (Styler et al 2014), 7 the SHARP Template Annotations (Savova et al 2012), the MiPACQ (Multi-source Integrated Platform for Answering Clinical Questions) (Albright et al 2013), the IxA-Med-GS (Oronoz et al 2015) or the Harvey corpus (Savkov et al 2016). Research challenges have also fuelled the annotation of resources or enrichment of available texts.…”

Natural Language Processing in Biomedicine: A Unified System Architecture Overview

“…This problem is exacerbated in the biomedical domain, where suitably qualified annotators can be both hard to find and prohibitively expensive [48,49].…”

Temporal Expression Recognition for Cell Cycle Phase Concepts in Biomedical Literature

“…However, this ongoing work on temporal evaluation is based on language data collected from the news. In the clinical domain, (Styler IV et al, Undated;Palmer and Pustejovsky, 2012;Albright et al, 2013) describe the THYME annotation project. The scope and language of temporality related to the cell cycle is different from that of both TempEval and the clinical domain, and supports (and demands) different types of reasoning, specifically related to cyclical time.…”