A GPT-2 Language Model for Biomedical Texts in Portuguese

Schneider, Elisa Terumi Rubel; Souza, João Vitor Andrioli de; Gumiel, Yohan Bonescki; Moro, Cláudia Maria Cabral; Paraíso, Emerson Cabrera

doi:10.1109/cbms52027.2021.00056

Cited by 24 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At its core, GPT is founded upon the innovative Transformer architecture, a model that has revolutionized the field by effectively capturing long-term dependencies within sequences, making it exceptionally well-suited for tasks involving language understanding and generation [ 45 , 46 ]. The GPT family has multiple versions: The GPT-2 model, with 1.5 billion parameters, is capable of generating extensive sequences of text while adapting to the style and content of arbitrary inputs [ 47 ]. Moreover, GPT-2 can also perform various NLP tasks, such as classification [ 47 ].…”

Section: Methodsmentioning

confidence: 99%

AE-GPT: Using Large Language Models to extract adverse events from surveillance reports-A use case with influenza vaccine adverse events

Li,

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Though Vaccines are instrumental in global health, mitigating infectious diseases and pandemic outbreaks, they can occasionally lead to adverse events (AEs). Recently, Large Language Models (LLMs) have shown promise in effectively identifying and cataloging AEs within clinical reports. Utilizing data from the Vaccine Adverse Event Reporting System (VAERS) from 1990 to 2016, this study particularly focuses on AEs to evaluate LLMs’ capability for AE extraction. A variety of prevalent LLMs, including GPT-2, GPT-3 variants, GPT-4, and Llama2, were evaluated using Influenza vaccine as a use case. The fine-tuned GPT 3.5 model (AE-GPT) stood out with a 0.704 averaged micro F1 score for strict match and 0.816 for relaxed match. The encouraging performance of the AE-GPT underscores LLMs’ potential in processing medical data, indicating a significant stride towards advanced AE detection, thus presumably generalizable to other AE extraction tasks.

show abstract

Section: Methodsmentioning

confidence: 99%

AE-GPT: Using Large Language Models to extract adverse events from surveillance reports-A use case with influenza vaccine adverse events

Li,

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Its successor, GPT-2, was also trained with a massive text data heap using a language-modeling task. However, unlike the previous version, it could generate longer text sequences consistent with human language [32].…”

Section: Openai and The Development Of Chatgptmentioning

confidence: 99%

“…Perceived importance (PIM) refers to students' general beliefs about technology and is linked to the cognitive component [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38]. The perceived importance of a technological tool affects the intention to use it in the future [40].…”

Section: Hypotheses From the Constructs Of The Cognitive Component Of...mentioning

confidence: 99%

Analysis College Students' attitude towards the use of ChatGPT in their academic activities: Effect of intent to use, verify information and responsible use

Enriquez,

Ballesteros,

Jordan

et al. 2023

Preprint

View full text Add to dashboard Cite

Background In recent years, the application of artificial intelligence (AI) in education has increased worldwide. The launch of ChatGPT - 3, posed great challenges for higher education, given its popularity among university students. The present study aimed to analyze the attitude of university students towards the use of ChatGPT in their academic activities. Method The study was oriented to the quantitative approach and had a non-experimental design of applied type. An online survey was administered to the 499 participants. Results The results demonstrated nine out of 12 supported hypotheses: importance (β=0.193*), ease of use (β=0.204**), risk (β=-0.104**), interest (β=0.238**), boredom (β=-0.145**), positive emotions (β=0.418***), acceptance (β=0.441***), intention to use frequently (β=0.509***), intention to verify information, and responsible use (β=0.806***). Implications of the findings from previous research are discussed, providing a unified theory of attitude towards ChatGPT-UTAC that empirically tests the underlying constructs in students' attitudes toward ChatGPT use. Conclusion Ease of use, intention to use frequently, acceptance, and intention to verify information influenced the behavioral intention to use ChatGPT responsibly. On the one hand, the study provides suggestions for HEIs to implement improvements in their educational curricula to take advantage of the potential benefits of AI and contribute to AI literacy.

show abstract

“…Models like the BERT, ELMO, and GPT-2 develop a rich context in terms of efficiency and predicting various interpretations. Schneider et al [103] proposed the GPT-2 to identify a Portuguese biomedical text. They used a fine-tuning approach for transfer learning; however, they manually annotated the public dataset for classification tasks.…”

Section: Gpt-2 Model For Contextual Word Embeddings (Cwe)mentioning

confidence: 99%

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

et al. 2023

View full text Add to dashboard Cite

In natural language processing, short-text semantic similarity (STSS) is a very prominent field. It has a significant impact on a broad range of applications, such as question–answering systems, information retrieval, entity recognition, text analytics, sentiment classification, and so on. Despite their widespread use, many traditional machine learning techniques are incapable of identifying the semantics of short text. Traditional methods are based on ontologies, knowledge graphs, and corpus-based methods. The performance of these methods is influenced by the manually defined rules. Applying such measures is still difficult, since it poses various semantic challenges. In the existing literature, the most recent advances in short-text semantic similarity (STSS) research are not included. This study presents the systematic literature review (SLR) with the aim to (i) explain short sentence barriers in semantic similarity, (ii) identify the most appropriate standard deep learning techniques for the semantics of a short text, (iii) classify the language models that produce high-level contextual semantic information, (iv) determine appropriate datasets that are only intended for short text, and (v) highlight research challenges and proposed future improvements. To the best of our knowledge, we have provided an in-depth, comprehensive, and systematic review of short text semantic similarity trends, which will assist the researchers to reuse and enhance the semantic information.

show abstract

A GPT-2 Language Model for Biomedical Texts in Portuguese

Cited by 24 publications

References 15 publications

AE-GPT: Using Large Language Models to extract adverse events from surveillance reports-A use case with influenza vaccine adverse events

AE-GPT: Using Large Language Models to extract adverse events from surveillance reports-A use case with influenza vaccine adverse events

Analysis College Students' attitude towards the use of ChatGPT in their academic activities: Effect of intent to use, verify information and responsible use

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

Contact Info

Product

Resources

About