A review of auditing techniques for the Unified Medical Language System

Abstract

Objective The study sought to describe the literature related to the development of methods for auditing the Unified Medical Language System (UMLS), with particular attention to identifying errors and inconsistencies of attributes of the concepts in the UMLS Metathesaurus. Materials and Methods We applied the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) approach by searching the MEDLINE database and Google Scholar for studies referencing the UMLS and any of several terms related to auditing, error detection, and quality assurance. A qualitative analysis and summarization of articles that met inclusion criteria were performed. Results Eighty-three studies were reviewed in detail. We first categorized techniques based on various aspects including concepts, concept names, and synonymy (n = 37), semantic type assignments (n = 36), hierarchical relationships (n = 24), lateral relationships (n = 12), ontology enrichment (n = 8), and ontology alignment (n = 18). We also categorized the methods according to their level of automation (ie, automated systematic, automated heuristic, or manual) and the type of knowledge used (ie, intrinsic or extrinsic knowledge). Conclusions This study is a comprehensive review of the published methods for auditing the various conceptual aspects of the UMLS. Categorizing the auditing techniques according to the various aspects will enable the curators of the UMLS as well as researchers comprehensive easy access to this wealth of knowledge (eg, for auditing lateral relationships in the UMLS). We also reviewed ontology enrichment and alignment techniques due to their critical use of and impact on the UMLS.

The UMLS knowledge sources at 30: indispensable to current research and applications in biomedical informatics

“…Two articles in this issue focus on this line of research. L. Zheng, He, et al review the published literature on auditing techniques applied to the unique semantic features of the UMLS Metathesaurus, 24 and F. Zheng et al introduce a transformation-based method for auditing IS-A hierarchies. 25…”

Extending import detection algorithms for concept import from two to three biomedical terminologies

“…Quality assurance is an important part of the ontology life cycle and has been widely studied [27][28][29][30][31]. Different studies have focused on different aspects such as structural relationships (e.g.…”

Quality assurance and enrichment of biological and biomedical ontologies and terminologies

“…In [ 7 ], Amith et al surveyed more recent quality assurance approaches for biomedical ontologies. In [ 8 ], Zheng et al performed a thorough review of methods for auditing the Unified Medical Language System (UMLS), as well as ontology enrichment and alignment techniques. Two special issues [ 9 , 10 ] have been organized and published to showcase the state of the art in auditing and quality assurance of biomedical terminologies and ontologies in 2009 and 2018, respectively.…”