Ascle—A Python Natural Language Processing Toolkit for Medical Text Generation: Development and Evaluation Study

Abstract

Background: Medical texts present significant domain-specific challenges, and manually curating these texts is a timeconsuming and labor-intensive process. Therefore, natural language processing (NLP) algorithms have been developed to automate text processing. In the biomedical field, there are various toolkits for text processing, which have greatly improved the efficiency of handling unstructured text. However, these existing toolkits tend to emphasize different perspectives, and the lack of generation capabilities in any of them leaves a significant void.Objective: This study introduces Ascle, a pioneering NLP toolkit designed for medical text generation. Ascle is tailored for biomedical researchers and clinical staff with an easy-to-use, all-in-one solution that requires minimal programming expertise. For the first time, Ascle provides four advanced and challenging generative functions: question-answering, text summarization, text simplification, and machine translation. Additionally, Ascle integrates 12 essential NLP functions, along with query and search capabilities for clinical databases. Methods:We fine-tuned 32 domain-specific language models and evaluated them thoroughly on 27 established benchmarks. Additionally, for the question-answering task, we develop a retrieval-augmented generation (RAG) framework for LLMs that incorporates a medical knowledge graph with ranking techniques to enhance the reliability of generated answers. Results:The fine-tuned models and RAG framework consistently enhanced text generation tasks. For example, the fine-tuned models improved the machine translation task by 20.27 in terms of BLEU score. In the question-answering task, the RAG framework raised the ROUGE-L score by 18% over the vanilla models.Conclusions: This study introduces the development and evaluation of Ascle, a user-friendly NLP toolkit designed for medical text generation. All code is publicly available via https://github.com/Yale-LILY/Ascle. All fine-tuned language models can be JMIR Preprints Yang et al