Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosis

Tu, Ruibo; Ma, Chao; Zhang, Cheng

doi:10.48550/arxiv.2301.13819

Cited by 3 publications

(3 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is achieved by extending the standard OWL ontology language to express conditional, probabilistic relationships between classes. Three extensions currently exist: BayesOWL [71] 3 , PROWL [72] 4 and OntoBayes [73]. BayesOWL is a framework which extends OWL capabilities for modeling and reasoning with uncertainty.…”

Section: H Prior Work In Causal Modeling Methodology and Generalizabi...mentioning

confidence: 99%

“…GPTs and other LLMs are great tools for natural language processing, but they have well-known limitations in establishing causality, thereby impacting their effectiveness [3], [4], [5]. This is rooted in how Artificial Intelligence (AI) and Machine Learning (ML) deal with pattern recognition in text rather than explicit causal mechanisms and direct cause-and-effect relationships.…”

Section: Original Manuscriptmentioning

confidence: 99%

See 1 more Smart Citation

A Custom GPT Health Recommender System Informed by Causal Bayesian Networks and Authoritative Ontologies (Preprint)

Hu,

Kerschberg

2024

Preprint

View full text Add to dashboard Cite

BACKGROUND ChatGPT and other large language models (LLMs) are trained on extensive text data; they learn patterns and associations within the training text without an inherent understanding of underlying causal mechanisms. Establishing causation necessitates controlled experiments and observations, and as of May 2024, ChatGPT lacks access to experimental data and the capacity to learn analytical models from data. Recent advancements from OpenAI enable the creation of custom Generative Pre-trained Transformers (GPT) models using their GPT Builder. These custom GPTs can be tailored with causal knowledge from Causal Bayesian Networks (CBN), thereby producing a knowledgeable, health-recommender system with causal expertise. This system is not only easily accessible for patients, clinicians and other users, but also has the potential to significantly improve healthcare outcomes. OBJECTIVE This paper presents a practical solution –a custom GPT model integrated with Causal Bayesian Networks (CBNs) informed by Authoritative Medical Ontologies (AMOs) as prior foundational knowledge. AMOs are robust biomedical ontologies that encapsulate the expert knowledge of their creators. By utilizing structured information contained within these ontologies, we can generate an informed CBN which can be used to improve a custom GPT as a health recommender system. These enhanced GPTs offer profound insights into cause-and-effect among co-morbid symptoms within the disease domain. METHODS To demonstrate our recommender system, we learn a CBN using NIMH data for patients of Alzheimer’s Disease and augment this with causal knowledge from the International Classification of Diseases Version 10 Clinical Modification (ICD-10-CM). We compute the CBN using the Max-Min Hill-Climbing (MMHC) algorithm. We generate two separate CBNs using MMHC to compare predictive accuracies between a baseline and a CBN modified with causal mechanisms from ICD-10-CM. Our previous research using this method has resulted in a modified CBN that reflects the causal claims in the AMOs and agrees with both the AMOs and the observational dataset. With this causal model, we build a custom GPT using OpenAI’s GPT Builder. RESULTS The custom GPT contains both potentially causal and correlations among symptoms, as well as conditional probabilities for these relationships. The GPT will also contain knowledge of causal mechanisms from ICD-10-CM, extended information regarding symptom variables, and references to existing literature regarding comorbid Alzheimer’s Disease symptoms. Furthermore, because the modified CBN model establishes potentially causal relationships among symptoms which can be verified in existing epidemiological research, we can verify that the custom GPT also establishes these causal relationships. This creates a GPT that agrees with the modified CBN, which is a representation of existing subject matter expertise in the disease domain. CONCLUSIONS To obtain our CBN, we’ve used a previous methodology which obtains ordered variable pairs from authoritative ontology ICD-10-CM as prior expertise. In our past research, a CBN that is learned using MMHC can be improved significantly by considering prior sources of knowledge, if the algorithm is modified appropriately. This source of prior knowledge can be validated in existing literature as a sequence of events in AD progression, specific causal mechanisms among comorbid symptoms, and conditional probabilities from a Bayesian Network. The resulting modified network provides insight into the causal relationships expressed in the AD data and takes advantage of the expertise and knowledge contained in the AMOs. Since inferring causality from a CBN does not exist in a vacuum, the relationships within the CBN, regardless of the strength of the conditional probabilities, must be explored further. A custom GPT extends the causal knowledge within a CBN with general knowledge of symptoms and diseases, providing a tool capable of suggesting causal inference based on the analysis of real patient data. With a LLM, uncertainty is present but reasoned with, as it is present but reasoned with in the prior, posterior, and likelihood in Bayes Theorem. Moving forward, we would like to explore using ChatGPT to produce ordered variable pairs and to automate the validation of potentially causal information.

show abstract

Section: H Prior Work In Causal Modeling Methodology and Generalizabi...mentioning

confidence: 99%

Section: Original Manuscriptmentioning

confidence: 99%

A Custom GPT Health Recommender System Informed by Causal Bayesian Networks and Authoritative Ontologies (Preprint)

Hu,

Kerschberg

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Tu et al (2023) used ChatGPT to evaluate its capability in answering causal discovery questions using a medical benchmark but found inconsistent testing results. They observed that ChatGPT has exceptional pro ciency in natural language conversation and can answer a wide range of questions but has limitations in understanding new concepts and achieving consistent performance.…”

mentioning

confidence: 99%

Automated Analysis of Causal Relationships in Customer Reviews

Piriyakul

2023

Preprint

View full text Add to dashboard Cite

As the volume of unstructured data on social media continues to grow, it's becoming increasingly important to have a proactive marketing strategy that can extract knowledge from this data. This study explores the use of ChatGPT for detecting causal relations and analyzing significant themes in order to build models for marketing analysis. Using 400 sample reviews and contemporary techniques, a causal graph was synthesized and tested, showing good model fit. All paths in the causal network were found to be significant except for the one from "Customer experience" to "Customer Advocacy." The system identified three serial mediators: "Exceptional hospitality" ➔ "Quality lodging" ➔ "Customer experience" ➔ "Enjoyable time" ➔ "Customer Advocacy," with an effect size of 0.0106. This research highlights the potential of linguistic data for developing mathematical models in marketing research and expands the scope of scientific inquiry in this field.

show abstract

Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-Based Causal Discovery

Susanti,

Färber

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosis

Cited by 3 publications

References 1 publication

A Custom GPT Health Recommender System Informed by Causal Bayesian Networks and Authoritative Ontologies (Preprint)

A Custom GPT Health Recommender System Informed by Causal Bayesian Networks and Authoritative Ontologies (Preprint)

Automated Analysis of Causal Relationships in Customer Reviews

Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-Based Causal Discovery

Contact Info

Product

Resources

About