ChatGPT’s inconsistent moral advice influences users’ judgment

Abstract

ChatGPT is not only fun to chat with, but it also searches information, answers questions, and gives advice. With consistent moral advice, it can improve the moral judgment and decisions of users. Unfortunately, ChatGPT’s advice is not consistent. Nonetheless, it does influence users’ moral judgment, we find in an experiment, even if they know they are advised by a chatting bot, and they underestimate how much they are influenced. Thus, ChatGPT corrupts rather than improves its users’ moral judgment. While these findings call for better design of ChatGPT and similar bots, we also propose training to improve users’ digital literacy as a remedy. Transparency, however, is not sufficient to enable the responsible use of AI.

Exploring the proficiency of ChatGPT-4: An evaluation of its performance in the Taiwan advanced medical licensing examination

“…While we hypothesize that ChatGPT's diminished performance in the July 2023 exams might stem from exposure to new test questions, it is also possible that its inherent inconsistencies contributed to the initially observed low scores in July 2023. 40 , 41 Third, we did not evaluate the appropriateness or logical consistency of ChatGPT's reasoning for each question. Fourth, we relied on official answers provided on the Ministry of Examination's website in Taiwan as the benchmark for correctness.…”

A User-friendly Approach for the Diagnosis of Diabetic Retinopathy Using ChatGPT and Automated Machine Learning

“…Additionally, ChatGPT incorporates a degree of randomness, resulting in variations in its responses even when faced with the same question asked repeatedly. 41 However, large language models like ChatGPT predict each subsequent word based on the preceding context. This allows for multitude of ways to express the same idea with different phrasings.…”

Opening a conversation on responsible environmental data science in the age of large language models

“…Ultimately, scientists may one day outsource the coding itself, but will still need to be trained in how to prompt AI tools appropriately (Zamfirescu-Pereira et al, 2023), how to assess the validity of their outputs (Passi and Vorvoreanu, n.d.; Zombies in the Loop? 2023), and to consider the societal implications and applications of these outputs (Tomašev et al, 2020;Krügel et al, 2023).…”