Automated classification of clinical trial eligibility criteria text based on ensemble learning and metric learning

Abstract

Background Eligibility criteria are the primary strategy for screening the target participants of a clinical trial. Automated classification of clinical trial eligibility criteria text by using machine learning methods improves recruitment efficiency to reduce the cost of clinical research. However, existing methods suffer from poor classification performance due to the complexity and imbalance of eligibility criteria text data. Methods An ensemble learning-based model with metric learning is proposed for eligibility criteria classification. The model integrates a set of pre-trained models including Bidirectional Encoder Representations from Transformers (BERT), A Robustly Optimized BERT Pretraining Approach (RoBERTa), XLNet, Pre-training Text Encoders as Discriminators Rather Than Generators (ELECTRA), and Enhanced Representation through Knowledge Integration (ERNIE). Focal Loss is used as a loss function to address the data imbalance problem. Metric learning is employed to train the embedding of each base model for feature distinguish. Soft Voting is applied to achieve final classification of the ensemble model. The dataset is from the standard evaluation task 3 of 5th China Health Information Processing Conference containing 38,341 eligibility criteria text in 44 categories. Results Our ensemble method had an accuracy of 0.8497, a precision of 0.8229, and a recall of 0.8216 on the dataset. The macro F1-score was 0.8169, outperforming state-of-the-art baseline methods by 0.84% improvement on average. In addition, the performance improvement had a p-value of 2.152e-07 with a standard t-test, indicating that our model achieved a significant improvement. Conclusions A model for classifying eligibility criteria text of clinical trials based on multi-model ensemble learning and metric learning was proposed. The experiments demonstrated that the classification performance was improved by our ensemble model significantly. In addition, metric learning was able to improve word embedding representation and the focal loss reduced the impact of data imbalance to model performance.

Multi-Label Classification in Patient-Doctor Dialogues With the RoBERTa-WWM-ext + CNN (Robustly Optimized Bidirectional Encoder Representations From Transformers Pretraining Approach With Whole Word Masking Extended Combining a Convolutional Neural Network) Model: Named Entity Study

“…We can only use the truncation method to preprocess text, that is, first truncation, tail truncation, and head to tail truncation, which adds some difficulty to the preliminary work. According to the work of Zeng et al, the base model did improve the accuracy rate by adjusting the downstream tasks [30]. For the single model, XLNET and RoBERTa were better than BERT and ERNIE, and the integration of multiple models will improve the model by 2.58% on average.…”

A clustering-based sampling method for miRNA-disease association prediction

“…In this work, we implemented an ensemble learning framework for miRNA-disease association prediction. Inspired by the previous research ( Chen et al, 2019b ; Dai et al, 2020 ; Sherazi et al, 2021 ; Wang et al, 2021 ; Zeng et al, 2021 ), we built the CSMDA through the following three stages: 1) construct multiple training subsets to increase the diversity of base classifiers by randomly sampling from ; 2) perform the random forest-based feature selection to reduce noise and redundant information in the high-dimensional feature space; 3) use soft voting strategy to integrate the prediction results of all base classifiers. The process of constructing the ensemble learning framework is shown in Figure 3 .…”

“…Here, let represent the number of training subsets. Take an unknown miRNA-disease pair as sample input, m base classifiers could produce m prediction result for the sample, and then the prediction results were integrated by the soft voting strategy ( Sherazi et al, 2021 ; Wang et al, 2021 ; Zeng et al, 2021 ). Specifically, the output of the th sample by soft voting was defined as follows: …”

Kinerja Komparatif Optimasi Algoritma Naive Bayes dalam Klasifikasi Teks untuk Uji Klinis Kanker

“…Kriteria kelayakan adalah strategi utama untuk melakukan penyaringan target dalam sebuah uji klinis. Klasifikasi otomatis teks kriteria kelayakan uji klinis dengan menggunakan metode pembelajaran mesin meningkatkan efisiensi rekrutmen untuk mengurangi biaya riset klinis [12]. Karena persyaratan kelayakan untuk uji klinis biasanya ditulis dalam teks bebas, pemrosesannya memerlukan interpretasi komputer [13] Beberapa penelitian telah dilakukan oleh para peneliti dalam mengklasifikasikan data klinis kanker.…”