AI Unreliable Answers: A Case Study on ChatGPT

Amaro, Isidro R.; Greca, A.; Francese, Rita; Tortora, Genoveffa; Tucci, Carlos

doi:10.1007/978-3-031-35894-4_2

Cited by 14 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results indicated that faculty demonstrated a higher level of knowledge than students. Yet, more than 40% of surveyed students and faculty expressed unwavering trust in the reliability of ChatGPT's responses, a perception that does not align with reality given that most AI-driven conversational models are prone to errors and biases (e.g., Amaro et al, 2023;Ray, 2023). There are two direct implications for this finding.…”

Section: Knowledgementioning

confidence: 99%

Exploring Students’ and Faculty’s Knowledge, Attitudes, and Perceptions Towards ChatGPT: A Cross-Sectional Empirical Study

Kamoun,

El Ayeb,

Jabri

et al. 2024

JITE:Research

View full text Add to dashboard Cite

Aim/Purpose: This study explores the Knowledge, Attitude, and Perception (KAP) towards ChatGPT among university students and faculty. It also examines the faculty’s readiness to cope with the challenges and leverage the opportunities presented by AI-powered conversational models. Background: Launched on November 30, 2022, ChatGPT took the world by storm with its capability to generate high-quality written expressions in a conversational manner. The reactions to this innovation varied, from enthusiasm regarding its potential to enrich students’ learning to concerns about its threat to students’ cognitive development and academic integrity. A systematic exploration of students’ and faculty’s KAP towards ChatGPT can play an important role in addressing the multifaceted dimensions of AI-driven conversational models. Methodology: This study employs a cross-sectional survey research design based on questionnaires distributed to 145 faculty members, as well as 855 undergraduate and graduate students at the ESPRIT School of Engineering and School of Business. The student sample was based on stratified and convenience sampling, while the faculty sample was based on a consensus sampling approach. Contribution: To the best of our knowledge, this is the first reported study that contributes to understanding the KAP of students and faculty towards ChatGPT, as well as the readiness of faculty to effectively adopt AI-driven conversational models. Furthermore, our research contributes to the body of knowledge by taking Vygotsky’s (1978) principle of social interaction and its role in promoting cognitive development to a new level by hypothesizing that if students were to acquire the competencies to actively engage with AI-driven chatbots in meaningful discussions and collaborative conversations, they might be able to develop some higher-order thinking skills further. Findings: Our results indicated that faculty demonstrated a higher level of ChatGPT knowledge than students and that more than 40% of surveyed students and faculty expressed some trust in the reliability of ChatGPT’s responses, a perception that does not align with reality. Faculty attitude towards ChatGPT was comparatively more reserved compared to that of students and showcased varying opinions. Furthermore, the surveyed faculty showcased a more negative perception of ChatGPT than students, and they expressed a greater degree of skepticism. Our research revealed that 63.4% of surveyed faculty reported that they lack the requisite training and resources to integrate ChatGPT into their pedagogical practices. Recommendations for Practitioners: HEIs should take appropriate measures to enhance students’ and faculty’s knowledge, attitude, and perception regarding ChatGPT to stimulate ethical, meaningful, innovative, and engaging interactions and learning experiences. Recommendation for Researchers: Our study has shed light on some moderating factors that shape the acceptance of AI-driven conversational models and some adoption barriers. It delves into the perceptions, biases, and misconceptions held by both students and faculty, thereby providing a basis for future investigations on the effective integration of AI-driven conversational models in higher education. Impact on Society: This research provides new insights that can harness the potential merits of ChatGPT in enhancing students’ learning while mitigating potential pitfalls. It suggests facilitating open forums and dialogues among students, faculty, employers, and other key stakeholders to debate the impact of AI-driven conversational models on students’ learning and faculty’s teaching and assessment. Future Research: We invite researchers to conduct cross-cultural studies on this topic while also taking into consideration a qualitative research design approach. Future research can also test the hypothesis that AI-driven conversational models inhibit critical thinking by facilitating the passive consumption of information.

show abstract

Section: Knowledgementioning

confidence: 99%

Exploring Students’ and Faculty’s Knowledge, Attitudes, and Perceptions Towards ChatGPT: A Cross-Sectional Empirical Study

Kamoun,

El Ayeb,

Jabri

et al. 2024

JITE:Research

View full text Add to dashboard Cite

show abstract

“…classes can often bear some resemblance to each other. However, ChatGPT's responses are sometimes unreliable, with errors detected as such intelligent answering machines can produce reasonable results but are incorrect or illogical (Amaro et al, 2023;Fitria, 2023). Therefore, it is vital to explore whether the students are aware of this AI technology's limitations and use it with caution.…”

Section: Introductionmentioning

confidence: 99%

Using ChatGPT in English Language Learning: A Study on I.T. Students’ Attitudes, Habits, and Perceptions

2024

ijte

View full text Add to dashboard Cite

In the realm of AI-driven education, it is pivotal to evaluate the viability of ChatGPT as a substitute for human teachers in English classrooms. This study aims to explore learners' behaviors, perceptions, and attitudes to ChatGPT usage in English language learning. Participants were 120 I.T. students in Vietnam – the Korea University of Information and Communication Technology and the University of Da Nang- who were learning English as a non-specialized subject. Data collection was conducted with multiple choices, a 4-point Likert scale questionnaire, and in-depth interviews. The findings highlight students' need for teacher's instruction and physical classroom despite recognizing ChatGPT’s efficacy for ESP vocabulary acquisition, translation, grammar checking, and paraphrasing. Students predominantly exploited ChatGPT to find instant solutions to English learning difficulties. The research underscores the importance of guiding learners to appropriately utilize ChatGPT, emphasizing the need for further investigation into plagiarism-detecting tools to mitigate potential misuse of the technology.

show abstract

Genie‐on‐demand: A custom AI chatbot for enhancing learning performance, self‐efficacy, and technology acceptance in occupational health and safety for engineering education

Hakim,

Paiman,

Rahman

2024

Comp Applic In Engineering

View full text Add to dashboard Cite

Occupational Health and Safety (OHS) education is essential for preparing engineering students to maintain safety standards and prevent workplace hazards. Traditional learning resources, such as textbooks, can be time‐consuming and inadequate for immediate, context‐specific queries. Advanced AI chatbots offer interactive and immediate feedback, but they often lack specificity and depend on users' prompting skills, which not all students possess. This study introduces “Genie‐on‐Demand,” a custom AI chatbot designed to address students' queries with precise, curriculum‐aligned responses. Educators can train the chatbot using specific learning materials by uploading PDFs, ensuring relevant and accurate answers. A quasi‐experimental study was conducted with 106 electrical engineering students divided into three groups: those using the custom chatbot, those using a conventional chatbot (ChatGPT), and those employing traditional learning methods. Results demonstrated that the custom chatbot significantly improved learning performance, self‐efficacy, and technology acceptance compared to other methods. Students reported increased confidence and effectiveness in using the custom chatbot as a learning assistant. This study highlights the potential of customized AI solutions in education, with versatile applications across various disciplines.

show abstract

AI Unreliable Answers: A Case Study on ChatGPT

Cited by 14 publications

References 11 publications

Exploring Students’ and Faculty’s Knowledge, Attitudes, and Perceptions Towards ChatGPT: A Cross-Sectional Empirical Study

Exploring Students’ and Faculty’s Knowledge, Attitudes, and Perceptions Towards ChatGPT: A Cross-Sectional Empirical Study

Using ChatGPT in English Language Learning: A Study on I.T. Students’ Attitudes, Habits, and Perceptions

Genie‐on‐demand: A custom AI chatbot for enhancing learning performance, self‐efficacy, and technology acceptance in occupational health and safety for engineering education

Contact Info

Product

Resources

About