2023
DOI: 10.2196/47479
|View full text |Cite
|
Sign up to set email alerts
|

Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument

Abstract: Background ChatGPT-4 is the latest release of a novel artificial intelligence (AI) chatbot able to answer freely formulated and complex questions. In the near future, ChatGPT could become the new standard for health care professionals and patients to access medical information. However, little is known about the quality of medical information provided by the AI. Objective We aimed to assess the reliability of medical information provided by ChatGPT. … Show more

Help me understand this report
View preprint versions

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

0
42
0

Year Published

2023
2023
2024
2024

Publication Types

Select...
6
3
1

Relationship

0
10

Authors

Journals

citations
Cited by 132 publications
(42 citation statements)
references
References 28 publications
0
42
0
Order By: Relevance
“…When disseminating information about cancer treatment and sexual health issues faced by cancer survivors, the generated chatbots functioned without refusing to answer, with or without training sources of medical guidelines. GPT responses have been noted to be as reliable as web searches and are closer to clinical guidelines, making it a promising tool to support medical communication 7 8. In this study, the GPT returned useful results comparable to the guidelines, not calling for excessive pessimism or optimism.…”
Section: Discussionmentioning
confidence: 63%
“…When disseminating information about cancer treatment and sexual health issues faced by cancer survivors, the generated chatbots functioned without refusing to answer, with or without training sources of medical guidelines. GPT responses have been noted to be as reliable as web searches and are closer to clinical guidelines, making it a promising tool to support medical communication 7 8. In this study, the GPT returned useful results comparable to the guidelines, not calling for excessive pessimism or optimism.…”
Section: Discussionmentioning
confidence: 63%
“…Second, ChatGPT struggled to interpret all the coherent laboratory tests [60], generating superficial and incorrect responses. Indeed, ChatGPT could generate overly general answers without citing original references [20,40,42].…”
Section: Resultsmentioning
confidence: 99%
“…This outcome underscores the imperative for exercising caution when solely relying on AI-generated medical information and the need for continuous evaluation, as others have noted [ 16 ]. However, in another study by Walker et al [ 17 ] aimed at evaluating the reliability of medical information provided by ChatGPT-4, multiple iterations of their queries executed through the model yielded a remarkable 100% internal consistency among the generated outputs [ 17 ]. Although promising, it should be noted that the queries used in their experiment consisted of direct single-sentence questions pertaining to specific hepatobiliary diagnoses.…”
Section: Discussionmentioning
confidence: 99%