A Conversational Agent to Improve Response Quality in Course Evaluations

Abstract

Recent advances in Natural Language Processing (NLP) bear the opportunity to design new forms of human-computer interaction with conversational interfaces. We hypothesize that these interfaces can interactively engage students to increase response quality of course evaluations in education compared to the common standard of web surveys. Past research indicates that web surveys come with disadvantages, such as poor response quality caused by inattention, survey fatigue or satisficing behavior. To test if conversational interfaces have a positive impact on the level of enjoyment and the response quality, we design an NLPbased conversational agent and deploy it in a field experiment with 127 students in our lecture and compare it with a web survey as a baseline. Our findings indicate that using conversational agents for evaluations are resulting in higher levels of response quality and level of enjoyment, and are therefore, a promising approach to increase the effectiveness of surveys in general.

ArgueTutor: An Adaptive Dialog-Based Learning System for Argumentation Skills

“…The development of PCAs dates back to the 1970s research stream of Intelligent Tutoring Systems (ITS) [3,63]. Similar to a human tutor, these systems can present instructions, ask questions [80] and provide immediate feedback [38]. ITS evolved from abstract entities with limited technological possibilities to systems that are able to interact with learners using multiple channels of communication, exhibit social skills and perform diferent roles, such as tutors [46], motivators or learning companions [30] as well as conducting course evaluations to assist teachers [80].…”

“…Similar to a human tutor, these systems can present instructions, ask questions [80] and provide immediate feedback [38]. ITS evolved from abstract entities with limited technological possibilities to systems that are able to interact with learners using multiple channels of communication, exhibit social skills and perform diferent roles, such as tutors [46], motivators or learning companions [30] as well as conducting course evaluations to assist teachers [80]. A beneft of using the technology of PCAs compared to traditional technology-enhanced argumentation learning systems is the increasing engagement of the students due to the dialog-based interaction of the learners with the PCA.…”

How the Wakamola chatbot studied a university community’s lifestyle during the COVID-19 confinement

“…The data collected is of particular interest for analysing lifestyle changes due to the exceptional circumstances. The Wakamola chatbot turned out to be useful to contact users in confinement and collect data on their weekly habits, and allowed us to obtain the mean population BMI and changes in weight and lifestyle that could be useful to guide preventive actions against overweight and obesity, while the information on dietary habits could help to study dietary patterns, 54,64 the relationship between nutrition and obesity and other diseases. 65,66 This chatbot could also be used not only to collect data but also as an interactive tool to promote home-based physical activity and healthy diets, help obesity screening, prevalence studies, and in monitoring confined populations.…”

Designing Adaptive Argumentation Learning Systems Based on Artificial Intelligence

“…One possible solution to this dilemma is using adaptive argumentation learning tools that provide individual feedback on the argumentation of students' texts, e.g., through adaptive writing support systems or pedagogical conversational agents. These intelligent tutoring systems (ITS) often imitate meaningful, individual instructor-learner interactions (e.g., [16]) and have been successfully used in adaptively supporting learners to conduct a task by mimicking the gold standard of human tutors (e.g., [43,46,49]). By using such systems, students would be able to learn autonomously and independently of the instructor, time, and place [33,42].…”