PubMed Text Similarity Model and its application to curation efforts in the Conserved Domain Database

Abstract

This study proposes a text similarity model to help biocuration efforts of the Conserved Domain Database (CDD). CDD is a curated resource that catalogs annotated multiple sequence alignment models for ancient domains and full-length proteins. These models allow for fast searching and quick identification of conserved motifs in protein sequences via Reverse PSI-BLAST. In addition, CDD curators prepare summaries detailing the function of these conserved domains and specific protein families, based on published peer-reviewed articles. To facilitate information access for database users, it is desirable to specifically identify the referenced articles that support the assertions of curator-composed sentences. Moreover, CDD curators desire an alert system that scans the newly published literature and proposes related articles of relevance to the existing CDD records. Our approach to address these needs is a text similarity method that automatically maps a curator-written statement to candidate sentences extracted from the list of referenced articles, as well as the articles in the PubMed Central database. To evaluate this proposal, we paired CDD description sentences with the top 10 matching sentences from the literature, which were given to curators for review. Through this exercise, we discovered that we were able to map the articles in the reference list to the CDD description statements with an accuracy of 77%. In the dataset that was reviewed by curators, we were able to successfully provide references for 86% of the curator statements. In addition, we suggested new articles for curator review, which were accepted by curators to be added into the reference list at an acceptance rate of 50%. Through this process, we developed a substantial corpus of similar sentences from biomedical articles on protein sequence, structure and function research, which constitute the CDD text similarity corpus. This corpus contains 5159 sentence pairs judged for their similarity on a scale from 1 (low) to 5 (high) doubly annotated by four CDD curators. Curator-assigned similarity scores have a Pearson correlation coefficient of 0.70 and an inter-annotator agreement of 85%. To date, this is the largest biomedical text similarity resource that has been manually judged, evaluated and made publicly available to the community to foster research and development of text similarity algorithms.

Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network

“…Finally, the proposed model achieves the best results by the self2self-attention on the three datasets except for the Spearman correlation coefficient on CDD-ref. The final Pearson correlation coefficient is increased to 0.661, approaching the official value 0.678 [33] on CDD-ref. The reasons for the better performance obtained by our model on the three corpora may be that the proposed self2self-attention not only helps our model to more precisely represent the semantic via self-attention in a single sentence, but enhance the sentence semantic representation through cross self-attention.…”

Interactive Self-Attentive Siamese Network for Biomedical Sentence Similarity

“…The direct reason may be that the interaction semantic information causes a slight amount of noise in this corpus. However, on CDD-ref corpus found in the same literature [36], ISA-SNN performs better. Therefore, we compare the difference between the two datasets according to the labeled score distribution, sentence length, and text quality.…”

Similarity Identification of Large-scale Biomedical Documents using Cosine Similarity and Parallel Computing

“…This phase is representative of algorithm TF*IDF. The TF*IDF-statistic short for term frequency times inverse the document frequency can extract keywords from a document by considering a single document and all documents from the corpus [2] [21]. The promising candidate for a keyword in a specific document if it shows up relatively often within the document and rarely in the rest of the…”

“…Large amounts of textual data could be collected as a part of the research, such as scientific literature, transcripts in the marketing and economic sectors, speeches in political discourse, such as presidential campaigns and inauguration speeches, and meeting transcripts [1]. PubMed dataset of MEDLINE also has grown enormously [2]. This large amount of textual information has created the problem of finding the relevance level between documents.…”

“…Keyword extraction is a vital algorithm to extract appropriate keywords that can easily choose which document to read to learn the relationship between documents in the form of document retrieval, web page retrieval, document clustering, summarization, text mining, and others. It will automatically identify terms that best describe the keywords of a document [2][9] [13]. Then, to obtain a suitable text relevance algorithm to demonstrate relevance calculation between two documents, many studies have been implemented the cosine similarity [9][14] [15].…”