Usability and User Experience of a Chat Application with Integrated Educational Chatbot Functionalities

Vukovac, Dijana Plantak; Horvat, Ana; Čižmešija, Antonela

doi:10.1007/978-3-030-77943-6_14

Cited by 6 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the structure of SUS comprises both usability and learnability [47], while in our study, this scale is used to specifically measure general usability. It must be noted that this scale is still very popular with numerous recently published studies that report its use for the evaluation of diverse technologies in various environments, including higher educa-tion, for instance, (a) concerning the use of the chat application Differ for communication with students [48], as well as (b) the evaluation of text generation by ChatGPT [49] (5 items; a sample item: "Bing Chat responds to my queries/commands as I expect it to. ").…”

Section: Instrumentmentioning

confidence: 99%

“…In this study, 63.2% of the students reported that they had used ChatGPT or other AI tools, but still, only 34.8% of them had used such tools "occasionally", "frequently", or "very often". The last aforementioned study revealed that the most commonly used AI tool by the students was ChatGPT (as reported by 48.9% of the respondents), while the following were found to be the most frequent reasons for their use of AI tools: clarifying questions and having subject-specific concepts explained (56.5%), research and literature studies (45.4%), translation (42.2%), text analysis, processing, and creation (39.3%), problem-solving and decision making (35.1%), and exam preparation (20.3%). This level of adoption of ChatGPT and similar tools by students in higher education, which can be rated as at least moderate, confirms the importance of evaluation of such tools for teaching and learning in the academic environment, taking into account previously mentioned purposes and other potential educational uses of AI tools.…”

Section: Introductionmentioning

confidence: 96%

See 1 more Smart Citation

Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI

Bubaš,

Čižmešija,

Kovačić

2023

Future Internet

View full text Add to dashboard Cite

After the introduction of the ChatGPT conversational artificial intelligence (CAI) tool in November 2022, there has been a rapidly growing interest in the use of such tools in higher education. While the educational uses of some other information technology (IT) tools (including collaboration and communication tools, learning management systems, chatbots, and videoconferencing tools) have been frequently evaluated regarding technology acceptance and usability attributes of those technologies, similar evaluations of CAI tools and services like ChatGPT, Bing Chat, and Bard have only recently started to appear in the scholarly literature. In our study, we present a newly developed set of assessment scales that are related to the usability and user experiences of CAI tools when used by university students, as well as the results of evaluation of these assessment scales specifically regarding the CAI Bing Chat tool (i.e., Microsoft Copilot). The following scales were developed and evaluated using a convenience sample (N = 126) of higher education students: Perceived Usefulness, General Usability, Learnability, System Reliability, Visual Design and Navigation, Information Quality, Information Display, Cognitive Involvement, Design Appeal, Trust, Personification, Risk Perception, and Intention to Use. For most of the aforementioned scales, internal consistency (Cronbach alpha) was in the range from satisfactory to good, which implies their potential usefulness for further studies of related attributes of CAI tools. A stepwise linear regression revealed that the most influential predictors of Intention to Use Bing Chat (or ChatGPT) in the future were the usability variable Perceived Usefulness and two user experience variables—Trust and Design Appeal. Also, our study revealed that students’ perceptions of various specific usability and user experience characteristics of Bing Chat were predominantly positive. The evaluated assessment scales could be beneficial in further research that would include other CAI tools like ChatGPT/GPT-4 and Bard.

show abstract

Section: Instrumentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI

Bubaš,

Čižmešija,

Kovačić

2023

Future Internet

View full text Add to dashboard Cite

show abstract

“…The data analysis revealed that chatbots could improve students' engagement in different learning contexts (Huang et al, 2022;Kohnke, 2022;Plantak Vukovac et al, 2021). Chatbots effectively provided immediate feedback, answered students' questions, and facilitated discussions, all contributing to student engagement.…”

Section: Data Analysis-student Engagementmentioning

confidence: 99%

Investigating the Impact of Chatbots in Different Learning Contexts on Student Engagement and Critical Thinking

Baskara

2023

JEEYAL

View full text Add to dashboard Cite

Technology integration in education has been a topic of interest for many years. Among those technologies, chatbots have been used in many educational settings. However, the impact of chatbots on students' engagement and critical thinking in different learning contexts has not been fully explored. This study aimed to investigate the impact of chatbots in different learning contexts on student engagement and critical thinking. A literature review was conducted to examine the previous studies on using chatbots in different learning contexts and relevant theoretical frameworks on student engagement and critical thinking in online and blended learning contexts. This study's results revealed that chatbots could improve students' engagement in different learning contexts and support the development of critical thinking skills. However, the design and features of chatbots should be aligned with the learning context to achieve optimal results. This study fills a gap in the literature by providing insight into how chatbot-mediated instruction may vary in different learning contexts and its possible impact on student engagement and critical thinking. The findings of this study have implications for chatbot design and instruction in different learning contexts, and it could be considered a valuable reference for educators, instructional designers, and researchers.

show abstract

Interface Design for Educational Chatbot to Increase Engagement for Online Learning: A Conceptual Design

Lei,

Dickson-Deane

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Usability and User Experience of a Chat Application with Integrated Educational Chatbot Functionalities

Cited by 6 publications

References 25 publications

Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI

Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI

Investigating the Impact of Chatbots in Different Learning Contexts on Student Engagement and Critical Thinking

Interface Design for Educational Chatbot to Increase Engagement for Online Learning: A Conceptual Design

Contact Info

Product

Resources

About