Clustering More than Two Million Biomedical Publications: Comparing the Accuracies of Nine Text-Based Similarity Approaches

Abstract

BackgroundWe investigate the accuracy of different similarity approaches for clustering over two million biomedical documents. Clustering large sets of text documents is important for a variety of information needs and applications such as collection management and navigation, summary and analysis. The few comparisons of clustering results from different similarity approaches have focused on small literature sets and have given conflicting results. Our study was designed to seek a robust answer to the question of which similarity approach would generate the most coherent clusters of a biomedical literature set of over two million documents.MethodologyWe used a corpus of 2.15 million recent (2004-2008) records from MEDLINE, and generated nine different document-document similarity matrices from information extracted from their bibliographic records, including titles, abstracts and subject headings. The nine approaches were comprised of five different analytical techniques with two data sources. The five analytical techniques are cosine similarity using term frequency-inverse document frequency vectors (tf-idf cosine), latent semantic analysis (LSA), topic modeling, and two Poisson-based language models – BM25 and PMRA (PubMed Related Articles). The two data sources were a) MeSH subject headings, and b) words from titles and abstracts. Each similarity matrix was filtered to keep the top-n highest similarities per document and then clustered using a combination of graph layout and average-link clustering. Cluster results from the nine similarity approaches were compared using (1) within-cluster textual coherence based on the Jensen-Shannon divergence, and (2) two concentration measures based on grant-to-article linkages indexed in MEDLINE.ConclusionsPubMed's own related article approach (PMRA) generated the most coherent and most concentrated cluster solution of the nine text-based similarity approaches tested, followed closely by the BM25 approach using titles and abstracts. Approaches using only MeSH subject headings were not competitive with those based on titles and abstracts.

A delineating procedure to retrieve relevant publication data in research areas: the case of nanocellulose

“…The development of publication-level classification systems is currently a subject of research. Boyack et al (2011) clustered a corpus of 2.15 million biomedical publications from Medline database (2004)(2005)(2006)(2007)(2008) which generated coherent and concentrated cluster solution of text-based similarity approaches based on keywords extracted from titles and abstracts. They found their approach more precise than the Medical Subject Headings.…”

Tensor-Based Semantically-Aware Topic Clustering of Biomedical Documents

“…Various clustering approaches such as suffix tree clustering were supplemented with ontological information in [26], whereas the accuracy of similarity metrics is discussed in [27]. A knowledge domain scheme based on bipartite graphs with MeSH is presented in [28].…”

Visualization and analysis of SCImago Journal & Country Rank structure via journal clustering

“…Ahlgren and Colliander (2009) tested the performance of the complete-linkage clustering method for visualizing and classifying a set of 43 documents of the journal 'Information Retrieval' according to several similarity measures based on document text, coupling and a hybrid approach. A combination of graphic presentations and clustering was also adopted by Boyack et al (2011), yet they applied average-link clustering on several similarity matrices based on significant words extracted from the title, abstract and keywords of the Medical Subject Headings (MeSH) of over 2 million scientific articles gathered from the Medline database. More recently, Waltman and Van Eck (2012) employed a new multilevel clustering algorithm on a direct citation network comprising nearly 10 million publications in order to create an automatic classification from clusters detected.…”