A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation

Abstract

Background A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence. Objective The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature. Methods Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores. Results Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022. Conclusions The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching. International Registered Report Identifier (IRRID) DERR1-10.2196/29398

Bibliometric Properties of Placebo Literature From the JIPS Database: A Descriptive Study

“…Prospective developments include the formulation of algorithms for the automated detection of relevant articles, e.g. via machine learning (34,35). The JIPS database itself is well-suited for this purpose, as it could be contrasted with the corpus of literature (i.e., all PubMed hits for "placebo", among other sources) from which it is drawn.…”

Application of Machine Learning Technology in Classical Music Education

“…It is not uncommon to replace the audio with video in today's high school music classes, and video editing and processing have become one teaching skills for music teachers besides professional skills. Video technology has the potential to render abstract music tangible and comprehensible for students (Abdelkader et al, 2021).…”

Introducing AI to the molecular tumor board: one direction toward the establishment of precision medicine using large-scale cancer clinical and biological information

“…On the other hand, the number of medical papers published to date is substantial, and it is a difficult task for humans to extract important information from them. Therefore, research has been conducted to efficiently extract useful information from medical papers using NLP [ 163 – 165 ], one of the AI techniques, and we introduce some recent representative results.…”