Automatic document indexing in large medical collections

Abstract

Term extraction relates to extracting the most characteristic or important terms (words or phrases) in a document. This information is commonly used for improving the accuracy of document indexing and retrieval in large text collections. It also allows for faster and better understanding of the contents of a document collection without first browsing through the contents of its documents. This paper presents AMTEX, an automatic term extraction method, specifically designed for the automatic indexing of documents in large medical collections such as MEDLINE, the premier bibliographic database of the U.S. National Library of Medicine (NLM). AMTEX combines MeSH, the terminological thesaurus resource of NLM, with a well-established method for extraction of domain terms, the C/NC-value method. The performance evaluation of various AMTEX configurations in the indexing task is measured against the current state-of-the-art, the MMTx method. The experimental results on a subset of MEDLINE documents demonstrate that AMTEX achieves better precision and recall than MMTx.

The AMTEx approach in the medical document indexing and retrieval application

“…i) C-Value threshold (T Cvalue ) for the term extraction, which in our initial experiments presented in [8] was set to its recommended value (T Cvalue = 1.5)…”

“…ii) Term expansion threshold (T Expansion ), whereupon we have experimented in our pilot small scale experiments with AMTEx [8] ;…”

“…The optimal value for each of these thresholds is not easy to determine, as each of these affects term recall at different stages of the AMTEx process [8]. A simple approach to this optimisation problem would be to consider only the threshold applied at the end of the process, the T FinalList .…”

“…In our pilot experiments with AMTEx v1 [8], in the Single-word Term Extraction step, we were attempting to find partial matches in MeSH, for all word constituents of an unmatched multi-word term. We have observed that single term insertion in our candidate list through that process produced worse results.…”

“…The limitations of MMTx in term extraction and in the UMLS Metathesaurus mapping have been analysed in detail in in the studies by Pratt and Yetisgen-Yildiz [6] and Divita et al [7]. Our experiments in a pilot study of MMTx and AMTEx on a small MEDLINE corpus showed that MMTx performance was low in precision and that its output greatly suffers by overgenerating terms, which diffuse the document concept leading to inaccurate indexing of MEDLINE documents [8], [9]. This reflects a design choice in MMTx, which attempts to favour recall by not focusing on MeSH, whereupon MEDLINE indexing has been based, and by incorporating a variant generation process which leads to term over-generation.…”