Medición de la obsolescencia de la información en revistas de salud pública de México

Gorbea-Portal, Salvador; Atrián-Salazar, Magda Luz

doi:10.24875/gmm.18003293

Cited by 3 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, the obsolescence index for medical textbooks of Internal Medicine is around five years, meaning that a fresh out-of-print book has five years of relevant information. On the other hand, with the time considered for writing, peer review, and journal submission, scientific papers may have an obsolescence index of one year [111,112]. This rapid turnover in medical knowledge underscores the importance of regular updates to AI models to ensure their recommendations remain current and clinically relevant.…”

Section: Limitations Of the Studymentioning

confidence: 99%

Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine

Guillen-Grima,

Guillen-Aguinaga,

Guillen-Aguinaga

et al. 2023

Clinics and Practice

View full text Add to dashboard Cite

The rapid progress in artificial intelligence, machine learning, and natural language processing has led to increasingly sophisticated large language models (LLMs) for use in healthcare. This study assesses the performance of two LLMs, the GPT-3.5 and GPT-4 models, in passing the MIR medical examination for access to medical specialist training in Spain. Our objectives included gauging the model’s overall performance, analyzing discrepancies across different medical specialties, discerning between theoretical and practical questions, estimating error proportions, and assessing the hypothetical severity of errors committed by a physician. Material and methods: We studied the 2022 Spanish MIR examination results after excluding those questions requiring image evaluations or having acknowledged errors. The remaining 182 questions were presented to the LLM GPT-4 and GPT-3.5 in Spanish and English. Logistic regression models analyzed the relationships between question length, sequence, and performance. We also analyzed the 23 questions with images, using GPT-4’s new image analysis capability. Results: GPT-4 outperformed GPT-3.5, scoring 86.81% in Spanish (p < 0.001). English translations had a slightly enhanced performance. GPT-4 scored 26.1% of the questions with images in English. The results were worse when the questions were in Spanish, 13.0%, although the differences were not statistically significant (p = 0.250). Among medical specialties, GPT-4 achieved a 100% correct response rate in several areas, and the Pharmacology, Critical Care, and Infectious Diseases specialties showed lower performance. The error analysis revealed that while a 13.2% error rate existed, the gravest categories, such as “error requiring intervention to sustain life” and “error resulting in death”, had a 0% rate. Conclusions: GPT-4 performs robustly on the Spanish MIR examination, with varying capabilities to discriminate knowledge across specialties. While the model’s high success rate is commendable, understanding the error severity is critical, especially when considering AI’s potential role in real-world medical practice and its implications for patient safety.

show abstract

Section: Limitations Of the Studymentioning

confidence: 99%

Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine

Guillen-Grima,

Guillen-Aguinaga,

Guillen-Aguinaga

et al. 2023

Clinics and Practice

View full text Add to dashboard Cite

show abstract

“…For instance, the obsolescence index for medical textbooks of Internal Medicine is around five years, meaning that a fresh out-of-print book has five years of information. On the other hand, with the time considered for writing, peer review, and journal submission, scientific papers may have an obsolescence index of one year [37,38]. This rapid turnover in medical knowledge underscores the importance of regular updates to AI models to ensure their recommendations remain current and clinically relevant.…”

Section: Limitations Of the Studymentioning

confidence: 99%

Evaluating ChatGPT Efficacy in Navigating the Spanish Medical Residency Entrance Examination (MIR): A New Horizon for AI in Clinical Medicine

Guillen-Grima,

Guillen-Aguinaga,

Guillen-Aguinaga

et al. 2023

Preprint

View full text Add to dashboard Cite

The rapid progress in artificial intelligence, machine learning, and natural language processing has led to the emergence of increasingly sophisticated large language models (LLMs) enabling their use in healthcare. The study assesses the performance of two LLMs: the GPT-3.5 and GPT-4 models in passing the medical examination for access to medical specialist training in Spain MIR. Our objectives included gauging the model's overall performance, analyzing discrepancies across different medical specialties, discerning between theoretical and practical questions, estimating error proportions, and assessing the hypothetical severity of errors committed by a physician. We studied the 2022 Spanish MIR examination after excluding those questions requiring image evaluations or having acknowledged errors. The remaining 182 questions were presented to the LLM ChatGPT4 and GPT-3.5 in Spanish and English. Logistic regression models analyzed the relationships between question length and question sequence d performance. GPT-4 outperformed GPT -3.5, scoring 86.81% in Spanish (p<0.001). English translations had a slightly enhanced performance. Among medical specialties, GPT-4 achieved a 100% correct response rate in several areas, with specialties like Pharmacology, ICU, and Infectious Diseases showing lower performance. The error analysis revealed that while a 13.2% error rate existed, gravest categories like "error requiring intervention to sustain life" and "error resulting in death" had a 0% rate. Conclusions: GPT-4 performs robustly on the Spanish MIR examination, varying its capability to discriminate knoweldge across specialties. While the model's high success rate is commendable, understanding the error severity is critical, especially when considering AI's potential role in real-world medical practice and its implication on patient safety.

show abstract

“…Another common approach is obsolescence studies of journals [21–23]. Obsolescence studies of geographical areas are also available [24,25]. Recently, obsolescence studies have focused on new data sources including citations in Wikipedia articles [26].…”

Section: Introductionmentioning

confidence: 99%

Obsolescence of the literature: A study of included studies in Cochrane reviews

Frandsen

Eriksen

Hammer

2021

Journal of Information Science

View full text Add to dashboard Cite

Ageing or obsolescence describes the process of declining use of a particular publication over time and can affect the results of a citation analyses as the length of citation window can change rankings. Obsolescence may not only vary across fields but also across subfields or sub-disciplines. The aim of this study is to determine the sub-disciplinary differences of obsolescence on a larger scale allowing for differences over time as well. The study presents the results of an analysis of 82,759 references across 53 healthcare and health policy topics. The references in this study are extracted from systematic reviews published from 2012 to 2016. The analyses of obsolescence include median citation age and mean citation age. This study finds that the median citation age and the mean citation age differ considerably across groups. For the latter indicator, an analysis of the confidence intervals confirms these differences. Using the subfield categorisation from Cochrane review groups, we found larger differences across subfields than in the citing half-lives published by Journal Citation Reports. Obsolescence is important to consider when setting the length of the citation windows. This study emphasises the vast differences across health sciences subfields. The length of the citation period is thus highly important for the results of a bibliometric evaluation or study covering fields with very varying obsolescence rates.

show abstract

Medición de la obsolescencia de la información en revistas de salud pública de México

Abstract: This study delineates an obsolescence profile for each journal: the aging factor and annual loss of usefulness are consistent with those shown by the leading public health journals in Latin America and Spain.

Cited by 3 publications

References 11 publications

Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine

Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine

Evaluating ChatGPT Efficacy in Navigating the Spanish Medical Residency Entrance Examination (MIR): A New Horizon for AI in Clinical Medicine

Obsolescence of the literature: A study of included studies in Cochrane reviews

Contact Info

Product

Resources

About