AI-Powered Renal Diet Support: Performance of ChatGPT, Bard AI, and Bing Chat

Abstract

Patients with chronic kidney disease (CKD) necessitate specialized renal diets to prevent complications such as hyperkalemia and hyperphosphatemia. A comprehensive assessment of food components is pivotal, yet burdensome for healthcare providers. With evolving artificial intelligence (AI) technology, models such as ChatGPT, Bard AI, and Bing Chat can be instrumental in educating patients and assisting professionals. To gauge the efficacy of different AI models in discerning potassium and phosphorus content in foods, four AI models—ChatGPT 3.5, ChatGPT 4, Bard AI, and Bing Chat—were evaluated. A total of 240 food items, curated from the Mayo Clinic Renal Diet Handbook for CKD patients, were input into each model. These items were characterized by their potassium (149 items) and phosphorus (91 items) content. Each model was tasked to categorize the items into high or low potassium and high phosphorus content. The results were juxtaposed with the Mayo Clinic Renal Diet Handbook’s recommendations. The concordance between repeated sessions was also evaluated to assess model consistency. Among the models tested, ChatGPT 4 displayed superior performance in identifying potassium content, correctly classifying 81% of the foods. It accurately discerned 60% of low potassium and 99% of high potassium foods. In comparison, ChatGPT 3.5 exhibited a 66% accuracy rate. Bard AI and Bing Chat models had an accuracy rate of 79% and 81%, respectively. Regarding phosphorus content, Bard AI stood out with a flawless 100% accuracy rate. ChatGPT 3.5 and Bing Chat recognized 85% and 89% of the high phosphorus foods correctly, while ChatGPT 4 registered a 77% accuracy rate. Emerging AI models manifest a diverse range of accuracy in discerning potassium and phosphorus content in foods suitable for CKD patients. ChatGPT 4, in particular, showed a marked improvement over its predecessor, especially in detecting potassium content. The Bard AI model exhibited exceptional precision for phosphorus identification. This study underscores the potential of AI models as efficient tools in renal dietary planning, though refinements are warranted for optimal utility.

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges

“…The authors were primarily affiliated with institutions in the United States (n=47 of 122 different countries identified per publication, 38.5%), followed by Germany (n=11/122, 9%), Turkey (n=7/122, 5.7%), the United Kingdom (n=6/122, 4.9%), China/Australia/Italy (n=5/122, 4.1%, respectively), and 24 (n=36/122, 29.5%) other countries. Most studies examined one or more applications based on the GPT-3.5 architecture (n=66 of 124 different LLMs examined per study, 53.2%) 13,26–29,31–34,36–40,42–49,52–54,56–61,63,65–67,71,72,74,75,77,78,81–89,91,92,94,95,97–100,102–104,106–109,111 , followed by GPT-4 (n=33/124, 26.6%) 13,25,27,29,30,34–36,41,43,50,51,54,55,58,61,64,68–70,74,76,79–81,83,87,89,90,93,96,98,99,101,105 , Bard (n=10/124, 8.1%; now known as Gemini) 33,48,49,55,73,74,80,87,94,99 , Bing Chat (n=7/124, 5.7%; now Microsoft Copilot) 49,51,55,73,94,99,110 , and other applications based on Bidirectional Encoder Representations from Transformers (BERT; n=4/124, 3...…”

“…Most studies examined one or more applications based on the GPT-3.5 architecture (n=66 of 124 different LLMs examined per study, 53.2%) 13,26–29,31–34,36–40,42–49,52–54,56–61,63,65–67,71,72,74,75,77,78,81–89,91,92,94,95,97–100,102–104,106–109,111 , followed by GPT-4 (n=33/124, 26.6%) 13,25,27,29,30,34–36,41,43,50,51,54,55,58,61,64,68–70,74,76,79–81,83,87,89,90,93,96,98,99,101,105 , Bard (n=10/124, 8.1%; now known as Gemini) 33,48,49,55,73,74,80,87,94,99 , Bing Chat (n=7/124, 5.7%; now Microsoft Copilot) 49,51,55,73,94,99,110 , and other applications based on Bidirectional Encoder Representations from Transformers (BERT; n=4/124, 3.2%) 13,83,84 , Large Language Model Meta-AI (LLaMA; n=3/124, 2.4%) 55 , or Claude by Anthropic (n=1/124, 0.8%) 55 . The majority of applications were p...…”

“…The evaluation of limitations in output data yielded 7 second-order codes concerning the non-reproducibility (n=38/89, 42.7%) 28,29,34,38,39,41,43,45,46,49,54–61,64,65,71–73,76,80,82,83,85,90,91,94,96,98,99,101,103–105 , non-comprehensiveness (n=78/89, 87.6%) 13,25,26,28–30,32–44,46,48–62,64,65,67–79,81–98,100,102–107,109–111 , incorrectness (n=78/89, 87.6%) 13,25–44,46,49–52,54–62,64–66,69–79,81–85,87–107,109–111 , (un-)safeness (n=39/89, 43.8%) 28,30,35,37,39,40,42–44,46,50,51,57–60,62,64,65,69,70,73,74,76,78–80,82,84,85,91…”

“…An overview of the presence of codes for each study is provided in Supplementary Section 3. The majority of articles investigated the use and feasibility of LLMs as medical chatbots (n=84/89, 94.4%) 13,[64][65][66]68,69,[98][99][100][101][102][103][104][105][106][107][108][109][110][111] , while fewer reports additionally or exclusively focused on the generation of patient information (n=19/89, 21.4%) 24,31,43,48,49,57,59,62,67,70,79,[88][89][90][91]97,102,106,107 , including clinical documentation such as informed consent forms (n=5/89, 5.6%) 43,67,91,97,102 and discharge instructions (n=1/89, 1.1%) 31 , or translation/summarization tasks of medical texts (n=5/89, 5.6%) 24,49,…”

Capacity for large language model chatbots to aid in orthopedic management, research, and patient queries

“…Current research has started to compare the performance of AI chatbots within medical subspecialties by assessing chatbot accuracy and highlighting opportunities for improvement. Chatbot utility is being explored as a strategy to aid physicians in tasks such as designing comprehensive diets for patients with chronic kidney disease, 5 serving as an adjunct in radiologic decision‐making, 6 and triaging ophthalmic conditions 7 . Chatbot performance is also being explored in aiding patients with frequently asked questions 8,9 about their cancer diagnosis, 10 or recent plastic surgery 11 .…”