Curious Questions About Covid-19 Pandemic with ChatGPT: Answers and Recommendations

TEKİNAY, Osman Nuri

doi:10.1007/s10439-023-03209-x

Cited by 6 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model might generate responses that align with popular beliefs or prevailing sentiments rather than relying on accurate and evidence-based information [ 25 ]. This has implications for its reliability as a source of information, especially in matters of public health and safety [ 26 ]. Inconsistency in responses poses another challenge.…”

Section: Discussionmentioning

confidence: 99%

Assessing ChatGPT's Diagnostic Accuracy and Therapeutic Strategies in Oral Pathologies: A Cross-Sectional Study

Uranbey,

Özbey,

Kaygısız

et al. 2024

Cureus

View full text Add to dashboard Cite

Background: The rapid adoption of artificial intelligence (AI) models in the medical field is due to their ability to collaborate with clinicians in the diagnosis and management of a wide range of conditions. This research assesses the diagnostic accuracy and therapeutic strategies of Chat Generative Pre-trained Transformer (ChatGPT) in comparison to dental professionals across 12 clinical cases. Methodology: ChatGPT 3.5 was queried for diagnoses and management plans for 12 retrospective cases. Physicians were tasked with rating the complexity of clinical scenarios and their agreement with the ChatGPT responses using a five-point Likert scale. Comparisons were made between the complexity of the cases and the accuracy of the diagnoses and treatment plans. Results: ChatGPT exhibited high accuracy in providing differential diagnoses and acceptable treatment plans. In a survey involving 30 attending physicians, scenarios were rated with an overall median difficulty level of 3, showing acceptable agreement with ChatGPT's differential diagnosis accuracy (overall median 4). Our study revealed lower diagnosis scores correlating with decreased treatment management scores, as demonstrated by univariate ordinal regression analysis. Conclusions: ChatGPT's rapid processing aids healthcare by offering an objective, evidence-based approach, reducing human error and workload. However, potential biases may affect outcomes and challenge less-experienced practitioners. AI in healthcare, including ChatGPT, is still evolving, and further research is needed to understand its full potential in analyzing clinical information, establishing diagnoses, and suggesting treatments.

show abstract

Section: Discussionmentioning

confidence: 99%

Assessing ChatGPT's Diagnostic Accuracy and Therapeutic Strategies in Oral Pathologies: A Cross-Sectional Study

Uranbey,

Özbey,

Kaygısız

et al. 2024

Cureus

View full text Add to dashboard Cite

show abstract

“…This study aims to assess ChatGPT's capability as a COVID-19 information service platform, providing the public with accurate and relevant information about the virus [17,18]. This research not only evaluates the performance of ChatGPT in disseminating COVID-19 information but also offers insights into other informational services related to epidemics.…”

Section: Introductionmentioning

confidence: 99%

Potential and Limitations of ChatGPT 3.5 and 4.0 as a Source of COVID-19 Information: Comprehensive Comparative Analysis of Generative and Authoritative Information

Wang,

Gao,

Liu

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

Background The COVID-19 pandemic, caused by the SARS-CoV-2 virus, has necessitated reliable and authoritative information for public guidance. The World Health Organization (WHO) has been a primary source of such information, disseminating it through a question and answer format on its official website. Concurrently, ChatGPT 3.5 and 4.0, a deep learning-based natural language generation system, has shown potential in generating diverse text types based on user input. Objective This study evaluates the accuracy of COVID-19 information generated by ChatGPT 3.5 and 4.0, assessing its potential as a supplementary public information source during the pandemic. Methods We extracted 487 COVID-19–related questions from the WHO’s official website and used ChatGPT 3.5 and 4.0 to generate corresponding answers. These generated answers were then compared against the official WHO responses for evaluation. Two clinical experts scored the generated answers on a scale of 0-5 across 4 dimensions—accuracy, comprehensiveness, relevance, and clarity—with higher scores indicating better performance in each dimension. The WHO responses served as the reference for this assessment. Additionally, we used the BERT (Bidirectional Encoder Representations from Transformers) model to generate similarity scores (0-1) between the generated and official answers, providing a dual validation mechanism. Results The mean (SD) scores for ChatGPT 3.5–generated answers were 3.47 (0.725) for accuracy, 3.89 (0.719) for comprehensiveness, 4.09 (0.787) for relevance, and 3.49 (0.809) for clarity. For ChatGPT 4.0, the mean (SD) scores were 4.15 (0.780), 4.47 (0.641), 4.56 (0.600), and 4.09 (0.698), respectively. All differences were statistically significant (P<.001), with ChatGPT 4.0 outperforming ChatGPT 3.5. The BERT model verification showed mean (SD) similarity scores of 0.83 (0.07) for ChatGPT 3.5 and 0.85 (0.07) for ChatGPT 4.0 compared with the official WHO answers. Conclusions ChatGPT 3.5 and 4.0 can generate accurate and relevant COVID-19 information to a certain extent. However, compared with official WHO responses, gaps and deficiencies exist. Thus, users of ChatGPT 3.5 and 4.0 should also reference other reliable information sources to mitigate potential misinformation risks. Notably, ChatGPT 4.0 outperformed ChatGPT 3.5 across all evaluated dimensions, a finding corroborated by BERT model validation.

show abstract

“…As for the field of infectious diseases, our research group has analyzed the most concerned questions about the use of ChatGPT in the infectious disease domain 5 , 6 . Additionally, several scholars also acquired answers and recommendations from ChatGPT on the frequently asked questions about COVID-19, such as spread, symptoms, diagnosis, treatment, vaccines, and pandemic management 7 . At the time point of WHO announced the end of COVID-19 PHEIC, we conducted an online survey using GPT-4 ( https://chat.openai.com/chat ).…”

mentioning

confidence: 99%

WHO declares end of COVID-19 global health emergency: lessons and recommendations from the perspective of ChatGPT/GPT-4

Cheng

et al. 2023

International Journal of Surgery

View full text Add to dashboard Cite

Curious Questions About Covid-19 Pandemic with ChatGPT: Answers and Recommendations

Cited by 6 publications

References 0 publications

Assessing ChatGPT's Diagnostic Accuracy and Therapeutic Strategies in Oral Pathologies: A Cross-Sectional Study

Assessing ChatGPT's Diagnostic Accuracy and Therapeutic Strategies in Oral Pathologies: A Cross-Sectional Study

Potential and Limitations of ChatGPT 3.5 and 4.0 as a Source of COVID-19 Information: Comprehensive Comparative Analysis of Generative and Authoritative Information

WHO declares end of COVID-19 global health emergency: lessons and recommendations from the perspective of ChatGPT/GPT-4

Contact Info

Product

Resources

About