Defining a state-of-the-art POS-tagging environment for Brazilian Portuguese clinical texts

Abstract

Purpose Natural language processing techniques are essential for unlocking patients' data from electronic health records. An important NLP task is the ability to recognize morphosyntactic information from the texts, a process called part-of-speech (POS) tagging. Currently, neural network architectures are the state-of-the-art method, although there is a lack of studies exploiting this approach within Brazilian Portuguese clinical texts. The objective of this study is to define a state-of-the-art POS-tagging environment for Brazilian Portuguese clinical texts. Methods We reviewed multiple neural network-based POS-tagging algorithms, and the Flair tool was selected due to its exceptional performance in the journalistic domain, as there is any specific algorithm to Portuguese clinical texts. We executed a normalization process on available corpora from multiple domains (two journalistic, one biomedical, one clinical, and a new corpus composed of all three of these). The Flair algorithm was trained with all corpora, generating five models, which were evaluated with all domains. Results The clinical model achieved 92.39% accuracy (previous POS-tagging clinical work reached 91.5%); the biomedical model achieved 97.9% accuracy. All the models were assessed on their own test set. Conclusion We developed a new state-of-the-art modeling environment for POS tagging of Brazilian Portuguese clinical texts and achieved comparable results to other state-of-the-art studies in journalistic contexts.

Developing a Transformer-based Clinical Part-of-Speech Tagger for Brazilian Portuguese

“…We achieved an F1 score and accuracy We also achieved superior results for the Mac-Morphos corpus using BERT-based models compared to models trained over other architectures reported in the literature, evidencing the impact of models based on the Transformer architecture, as BERT-based models, in NLP tasks. Regarding architectures: [9] used FLAIR; [17] We found that the lack of standardization of clinical texts directly impacted the POStagger-BERTimbau and POStagger-BioBERTpt models' results. Both models were trained with Mac-Morpho, a news texts corpora, and may not be robust enough to deal with some aspects particular to the clinical domain, such as extensive use of abbreviations and acronyms, uppercase terms, domain-specific vocabulary, flexible formatting, and atypical grammatical constructions [18].…”

“…We randomly selected 50 sentences containing between 6 and 15 tokens, which were manually POS-annotated by a human linguist, referred to in this paper as human annotation. For comparison purposes, we also evaluated two clinical POS-tagger models, trained with Flair [9] and Spacy [10], referred to in this paper as Flair and Spacy. For evaluation, we separated words pertaining to distinct functions in contracted forms in Portuguese, such as the contraction of a preposition with an article (eg "NA" was separated into "EM + A").…”

“…All the sentences were submitted to the models and, based on their output, we calculated the number of hits and errors of each model (accuracy), both as a general score and a score by category. In a post-processing stage, a) we tagged all the symbols in the models' outputs as SYM, since this tag is common in clinical texts but was not learned by the models, and b) we normalized tags using the tagset proposed by Universal Dependencies 3 , as each model has its own tagset (we used the same structure of the work of [9]). In addition, we performed the following experiments: 1) Sentence evaluation with the text in their actual typed form; 2) Sentence evaluation in lowercase; and 3) Automatic tagging of all numbers as NUM and sentences in lowercase.…”

“…We with the same clinical corpora used in [9] and [10] to compare with the current Flair and Spacy models, and further investigate the role of the Transformer architecture in POStagger tasks.…”

“…However, when it comes to the clinical domain, we have found few research initiatives targeting Portuguese. Worth highlighting are the study by [8], which trained a POS-tagger on clinical texts written in Portuguese; the study by [9], which trained a clinical POS-tagger model with Flair; and the study by [10], which created a Brazilian Portuguese corpus containing clinical texts annotated with morphological and syntactic information. To the best of our knowledge, no research has addressed Portuguese clinical POS-tagging using the Transformer architecture [11], which is currently responsible for the state-of-the-art for several NLP tasks.…”