Text dialogue analysis Based ChatGPT for Primary Screening of Mild Cognitive Impairment

Abstract

Background: AI models tailored to diagnose cognitive impairment have shown excellent results. However, it is unclear whether large linguistic models can rival specialized models by text alone. Objectives: We would explore the effectiveness of ChatGPT for primary screening of mild cognitive impairment (MCI) and standardize the design steps and components of the prompt. Methods: We obtained 174 participants from the DementiaBank screening and classified 70% of them into the training set and 30% into the test set. Three dimensions of variables were incorporated, including: vocabulary, syntax and grammar, and semantics. These variables were generated from published studies and statistical analyses. We used R 4.3.0. for the analysis of variables and diagnostic indicators. Results: The final variables included by published studies included: word frequency and word ratio, phrase frequency and phrase ratio, lexical complexity, syntactic complexity, grammatical components, semantic density, and semantic coherence; variables included in the analysis included: tip-of-the-tongue phenomenon (P < 0.001), difficulty with complex ideas (P < 0.001), and memory issues (P < 0.001). The final GPT4 model achieved the sensitivity (SEN) of 0.8636, specificity (SPE) of 0.9487 and area under the curve (AUC) of 0.9062 on the training set; on the test set, the SEN, SPE and AUC reached 0.7727, 0.8333 and 0.8030, respectively. The prompt consisted of five main parts, including character setting, scoring system setting, indicator setting, output setting, and explanatory information setting. Conclusion: ChatGPT was effective in primary screening of participants with possible MCI. Improved standardization of prompts by professional clinicians would also improve the performance of the model. It is important to note that ChatGPT is not a substitute for a clinician making a diagnosis.

Large Language Models to process, analyze, and synthesize biomedical texts – a scoping review

“…Notably, they have also been applied in innovative ways, such as generating symptom definitions to enrich datasets and improve annotator precision, as well as for summarizing radiology reports, potentially aiding clinical decision-making [87, 88]. ChatGPT’s interactive capabilities have also been explored for preliminary screening in cases of mild cognitive impairment [89].…”

Shaping the Future of Older Adult Care: ChatGPT, Advanced AI, and the Transformation of Clinical Practice

“…Through a sense of virtual companionship, connections, and nonjudgmental emotional support, ChatGPT can also help address social isolation and loneliness in older adults [ 11 , 12 ]. Creative applications of generative AI to advance health care for older adults, including remote health monitoring, mental health support, and personalized prevention of cognitive decline, have been increasingly explored in the literature and are expected to demonstrate effects and impacts in the future [ 13 - 16 ].…”

GPT-4 and Neurologists in Screening for Mild Cognitive Impairment in the Elderly: A Comparative Analysis Study

“…We included a total of 174 subjects from the DementiaBank English Protocol Delaware Corpus [16] and the DementiaBank English Pitt Corpus [17,18]. 66 MCI subjects and 108 CN subjects were included [21]. Step 2 was devised based on feedback data acquired from Feedback 2, which was utilized to generate entirely new prompts.…”

“…Subsequently, GPT-4 gained the capability to autonomously design prompts, resulting in the creation of three distinct prompts that were later amalgamated. Moreover, the culminated in the development of the GPT-4 Model [21].…”