Personalized Medicine in Urolithiasis: AI Chatbot-Assisted Dietary Management of Oxalate for Kidney Stone Prevention

Aiumtrakul, Noppawit; Thongprayoon, Charat; Arayangkool, Chinnawat; Vo, Kristine B.; Wannaphut, Chalothorn; Suppadungsuk, Supawadee; Krisanapan, Pajaree; Garcia Valencia, Oscar A.; Qureshi, Fawad; Miao, Jing; Cheungpasitporn, Wisit

doi:10.3390/jpm14010107

Cited by 12 publications

(5 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there are many administrative tasks in healthcare that are often labor-intensive, requiring manual input and contributing to physician burnout 41 . Particularly, areas such as assigning provider billing codes (1 study), writing prescriptions (1 study), generating clinical referrals (3 studies), and clinical note-taking (4 studies); all of which remain under-researched and could greatly benefit from a systematic evaluation of using LLMs for those tasks 38 39 50 51 .…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

Bedi,

Liu,

Orr-Ewing

et al. 2024

Preprint

View full text Add to dashboard Cite

Importance: Large Language Models (LLMs) can assist in a wide range of healthcare-related activities. Current approaches to evaluating LLMs make it difficult to identify the most impactful LLM application areas. Objective: To summarize the current evaluation of LLMs in healthcare in terms of 5 components: evaluation data type, healthcare task, Natural Language Processing (NLP)/Natural Language Understanding (NLU) task, dimension of evaluation, and medical specialty. Data Sources: A systematic search of PubMed and Web of Science was performed for studies published between 01-01-2022 and 02-19-2024. Study Selection: Studies evaluating one or more LLMs in healthcare. Data Extraction and Synthesis: Three independent reviewers categorized 519 studies in terms of data used in the evaluation, the healthcare tasks (the what) and the NLP/NLU tasks (the how) examined, the dimension(s) of evaluation, and the medical specialty studied. Results: Only 5% of reviewed studies utilized real patient care data for LLM evaluation. The most popular healthcare tasks were assessing medical knowledge (e.g. answering medical licensing exam questions, 44.5%), followed by making diagnoses (19.5%), and educating patients (17.7%). Administrative tasks such as assigning provider billing codes (0.2%), writing prescriptions (0.2%), generating clinical referrals (0.6%) and clinical notetaking (0.8%) were less studied. For NLP/NLU tasks, the vast majority of studies examined question answering (84.2%). Other tasks such as summarization (8.9%), conversational dialogue (3.3%), and translation (3.1%) were infrequent. Almost all studies (95.4%) used accuracy as the primary dimension of evaluation; fairness, bias and toxicity (15.8%), robustness (14.8%), deployment considerations (4.6%), and calibration and uncertainty (1.2%) were infrequently measured. Finally, in terms of medical specialty area, most studies were in internal medicine (42%), surgery (11.4%) and ophthalmology (6.9%), with nuclear medicine (0.6%), physical medicine (0.4%) and medical genetics (0.2%) being the least represented. Conclusions and Relevance: Existing evaluations of LLMs mostly focused on accuracy of question answering for medical exams, without consideration of real patient care data. Dimensions like fairness, bias and toxicity, robustness, and deployment considerations received limited attention. To draw meaningful conclusions and improve LLM adoption, future studies need to establish a standardized set of LLM applications and evaluation dimensions, perform evaluations using data from routine care, and broaden testing to include administrative tasks as well as multiple medical specialties. Keywords: Large Language Models, Generative Artificial Intelligence, Healthcare, Dimensions of Evaluation, Evaluation Metrics.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The process by which a healthcare provider, typically a physician or other qualified medical professional, orders medications or treatments for a patient Prescription of kidney stone prevention treatment (Alumtrakul et al) 39…”

Section: Writing Prescriptionsmentioning

confidence: 99%

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

Bedi,

Liu,

Orr-Ewing

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The Ho:YAG laser remains the gold standard for intervention, recommended by the European Association of Urology Guidelines [ 67 ]. The incorporation of artificial intelligence (AI) has further enhanced stone treatment by aiding in diagnosis, treatment planning, and prediction of kidney stone outcomes [ 36 , 68 ]. These advancements may have raised public awareness.…”

Section: Discussionmentioning

confidence: 99%

Global Trends in Kidney Stone Awareness: A Time Series Analysis from 2004–2023

Aiumtrakul,

Thongprayoon,

Suppadungsuk

et al. 2024

Clinics and Practice

Self Cite

View full text Add to dashboard Cite

Background: Despite the prevalence and incidence of kidney stones progressively increasing worldwide, public awareness of this condition remains unclear. Understanding trends of awareness can assist healthcare professionals and policymakers in planning and implementing targeted health interventions. This study investigated online search interest in “kidney stone” by analyzing Google Trends, focusing on stationarity of the trends and predicting future trends. Methods: We performed time series analysis on worldwide Google monthly search data from January 2004 to November 2023. The Augmented Dickey–Fuller (ADF) test was used to assess the stationarity of the data, with a p-value below 0.05 indicating stationarity. Time series forecasting was performed using the autoregressive integrated moving average to predict future trends. Results: The highest search interest for “kidney stone” (score 100) was in August 2022, while the lowest was in December 2007 (score 36). As of November 2023, search interest remained high, at 92. The ADF test was significant (p = 0.023), confirming data stationarity. The time series forecasting projected continued high public interest, likely reflecting ongoing concern and awareness. Notably, diverse regions such as Iran, the Philippines, Ecuador, the United States, and Nepal showed significant interest, suggesting widespread awareness of nephrolithiasis. Conclusion: This study highlighted that “kidney stone” is a consistently relevant health issue globally. The increase and stationarity of search trends, the forecasted sustained interest, and diverse regional interest emphasize the need for collaborative research and educational initiatives. This study’s analysis serves as a valuable tool for shaping future healthcare policies and research directions in addressing nephrolithiasis related health challenges.

show abstract

“…Additionally, there is evidence of ChatGPT’s ability to create discharge summaries and operative reports [ 40 , 41 ], record patient histories of present illness [ 42 ], and enhance the documentation process for informed consent [ 43 ], although its effectiveness requires further improvement. Within the specific scope of our research in nephrology, we have explored the use of chatbots in various areas such as innovating personalized patient care, critical care in nephrology, and kidney transplant management [ 44 ], as well as dietary guidance for renal patients [ 45 , 46 ] and addressing nephrology-related questions [ 47 ].…”

Section: Introductionmentioning

confidence: 99%

“…We also compared the relevance of ChatGPT, Bing Chat, and Bard AI in nephrology literature searches, with accuracy rates of only 38%, 30%, and 3%, respectively [ 59 ]. The occurrence of hallucinations during the literature searches, combined with the suboptimal accuracy in responding to nephrology inquiries [ 47 ] and correctly identifying oxalate, potassium, and phosphorus in diets [ 45 , 46 ], compromises the reliability or dependability of LLM outputs, raising significant concerns about their practical application. In critical areas like healthcare decision making, the impact of such inaccuracies is considerably heightened, highlighting the need for models that are more reliable and precise.…”

Section: Introductionmentioning

confidence: 99%

Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications

Miao,

Thongprayoon,

Suppadungsuk

et al. 2024

Medicina

Self Cite

View full text Add to dashboard Cite

The integration of large language models (LLMs) into healthcare, particularly in nephrology, represents a significant advancement in applying advanced technology to patient care, medical research, and education. These advanced models have progressed from simple text processors to tools capable of deep language understanding, offering innovative ways to handle health-related data, thus improving medical practice efficiency and effectiveness. A significant challenge in medical applications of LLMs is their imperfect accuracy and/or tendency to produce hallucinations—outputs that are factually incorrect or irrelevant. This issue is particularly critical in healthcare, where precision is essential, as inaccuracies can undermine the reliability of these models in crucial decision-making processes. To overcome these challenges, various strategies have been developed. One such strategy is prompt engineering, like the chain-of-thought approach, which directs LLMs towards more accurate responses by breaking down the problem into intermediate steps or reasoning sequences. Another one is the retrieval-augmented generation (RAG) strategy, which helps address hallucinations by integrating external data, enhancing output accuracy and relevance. Hence, RAG is favored for tasks requiring up-to-date, comprehensive information, such as in clinical decision making or educational applications. In this article, we showcase the creation of a specialized ChatGPT model integrated with a RAG system, tailored to align with the KDIGO 2023 guidelines for chronic kidney disease. This example demonstrates its potential in providing specialized, accurate medical advice, marking a step towards more reliable and efficient nephrology practices.

show abstract

Personalized Medicine in Urolithiasis: AI Chatbot-Assisted Dietary Management of Oxalate for Kidney Stone Prevention

Cited by 12 publications

References 46 publications

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

Global Trends in Kidney Stone Awareness: A Time Series Analysis from 2004–2023

Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications

Contact Info

Product

Resources

About