Medical Text Prediction and Suggestion Using Generative Pretrained Transformer Models with Dental Medical Notes

Abstract

Background: Generative pre-trained transformer (GPT) models are one of the latest large pre-trained natural language processing (NLP) models, which enables model training with limited datasets, and reduces dependency on large datasets which are scarce and costly to establish and maintain. There is a rising interest to explore the use of GPT models in healthcare. Objective: We investigate the performance of GPT-2 and GPT-Neo models for medical text prediction using 374,787 free-text dental notes. Methods: We fine-tune pre-trained GPT-2 and GPT-Neo models for next word prediction on a dataset of over 374,000 manually written sections of dental clinical notes. Each model was trained on 80% of the dataset, validated on 10%, and tested on the remaining 10%. We report model performance in terms of next word prediction accuracy and loss. Additionally, we analyze the performance of the models on different types of prediction tokens for categories. For comparison, we also fine-tuned a non-GPT pre-trained neural network model, XLNet (large), for next word prediction. We annotate each token in 100 randomly sampled notes by category (e.g. Names, Abbreviations, Clinical Terms, Punctuation, etc.) and compare the performance of each model by token category. Results: Models present acceptable accuracy scores (GPT-2: 76%, GPT-Neo: 53%), and the GPT-2 model also performs better in manual evaluations, especially for names, abbreviations, and punctuation. Both GPT models outperformed XLNet in terms of accuracy. The results suggest that pre-trained models have the potential to assist medical charting in the future. We share the lessons learned, insights, and suggestions for future implementations. Conclusion: The results suggest that pre-trained models have the potential to assist medical charting in the future. Our study presented one of the first implementations of the GPT model used with medical notes.

Integration and Assessment of ChatGPT in Medical Case Reporting: A Multifaceted Approach

“…The healthcare sector has long been a focal point for the application of artificial intelligence, and the recent introduction of ChatGPT has sparked a flurry of diverse applications [15][16][17][18][19][20][21]. The integration of ChatGPT in medical case report writing has unearthed a number of considerations, extending from the assessment to the broader implications of artificial intelligence in medical documentation.…”

“…Beyond its application by the general populace and in business enterprises [4], the healthcare sector has exhibited a great interest in exploring its potential [5][6][7][8][9][10][11][12][13] and has explored the applicability of ChatGPT in the context of medical record management, including medical records [14,15], surgical reports [16,17], radiology findings [18][19][20], and discharge summaries [17,21] in the realm of clinical practice. Traditional medical record composition requires the meticulous cross-referencing of textual content scattered across disparate files and the systematic organization of inpatient progress along chronological timelines.…”

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

“…However, the results indicate acceptable performance when using NER and dependency parsing through open-source and hybrid NLP models. The performance of the pipeline may increase over time with improvements in automatic speech recognition and text prediction and suggestion methods (methods that also use NLP models that are not covered within the scope of this study) [29][30][31]. However, in this study, the pipeline performance was potentially affected by the transcription errors or typing errors existing in the data set (n=16, 18% of 87 notes had at least one error; errors have not been corrected to contain real-world data features).…”

Year 2022 in Medical Natural Language Processing: Availability of Language Models as a Step in the Democratization of NLP in the Biomedical Area

“…More recently, the GPT (Generative Pretrained Transformer) models [72] have been proposed for various generation tasks. These models are also coming into the clinical domain but with only few works published in 2022: creation of BioGPT (Generative Pre-trained Transformer for biomedical text generation and mining) [73], prediction and suggestion of medical text in dental medical notes [74], challenges for GPT-3 in ophthalmology [75]. This generative model proposes text on the basis of the training corpora.…”