Burkitt Lymphoma

Hoelzer, Dieter; Burmeister, Thomas

doi:10.1007/978-1-62703-408-1_14

Cited by 1 publication

(1 citation statement)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, studies evaluating LLMs response capabilities in the eld of adolescent growth and development have not been found to date. [18][19] The 49th Statistical Report on Internet Development in China, released by the China Internet Network Information Center (CNNIC), reveals that around three-quarters of internet users in China use the internet to look up healthrelated information, with 84.5% of those in need of health advice turning to online sources. This trend is particularly noticeable in the area of adolescent growth and development, where both patients and their parents often rely on internet resources.…”

Section: Implications Of All the Available Evidencementioning

confidence: 99%

Benchmarking Large Language Models in Adolescent Growth and Development: A Comparative Analysis of Claude2, ChatGPT-3.5, and Google Bard

Li,

Song,

2024

Preprint

View full text Add to dashboard Cite

Background: Significant attention has been drawn to large-scale language models (LLMs) for their ability to generate responses that are both contextually relevant and reminiscent of human conversation. Yet, the precision of these models in specialized medical fields, particularly those pertaining to adolescent health, remains largely unexamined. Online searches for information about common health issues during adolescent developmental stages are frequent among patients and their families. In this context, our research evaluates how effectively three different LLMs - Claude2, ChatGPT-3.5, and Google Bard - handle typical inquiries concerning adolescent growth and health development. Methods: Our research involved gathering 100 frequently asked questions about adolescent growth and health issues, divided into 10 typical disorder categories: Attention Deficit, Tics, Developmental Delays, Autism Spectrum, Anxiety, Anorexia, Obsessive-Compulsive Disorder, Sleep Issues, Early Puberty, and Depressive Disorders. These questions were then posed to various large language models. A pediatric specialist evaluated the models' answers using a detailed four-tier system (ranging from Poor to Very Good) for accuracy. To ensure consistency, these assessments were revisited and verified at various intervals. High-scoring responses ('Good' or above) were examined closely for their compliance with medical ethics, treatment guidelines, and diagnostic procedures. In contrast, responses that scored lowest ('Poor') were subject to in-depth review, leading to recommendations for minor modifications based on straightforward query adjustments and online medical resources. These revised responses were then re-evaluated to measure any improvements in accuracy. Findings: Our study analyzed the performance of different models in adolescent growth and development issues. Claude2 was the top performer, with an average score of 3.54 and a standard deviation of 0.501. ChatGPT-3.5 was close behind, scoring an average of 3.44 and a standard deviation of 0.519. Human raters and Google Bard scored lower, at 2.60 and 2.49 respectively, with larger standard deviations. The one-way ANOVA showed significant differences (F-value 64.692, P-value 4.64e-34), particularly in areas like 'Attention Deficit Disorder', 'Developmental Delay', and 'Depression', where Claude2 and ChatGPT-3.5 outperformed others. The Pearson Chi-Square test (χ² value 117.758, P-value 2.35e-25) confirmed their accuracy and consistency. In self-correction abilities, Claude2, ChatGPT-3.5, and Bard scored 3.3, 3.0, and 2.4, respectively, for simple query-based corrections. For web-based medical self-corrections, the scores improved to 3.8, 3.5, and 3.7. The Pearson Chi-Square tests showed significant improvements for all models (Claude2 P-value 0.0241, ChatGPT-3.5 P-value 0.0150, Bard P-value 0.000017), with Bard showing the most significant improvement. This indicates that web-based medical correction methods significantly enhance performance in complex queries for all LLM chatbots. Interpretation: Our findings underscore the potential of Large Language Models (LLMs), particularly Claude2, in providing accurate and comprehensive responses to queries related to adolescent growth and development. The continual strategies and evaluations to enhance the accuracy of LLMs remain crucially important.

show abstract

Section: Implications Of All the Available Evidencementioning

confidence: 99%