One important problem in MOOCs is the lack of personalized support from teachers. Conversational agents arise as one possible solution to assist MOOC learners and help them to study. For example, conversational agents can help review key concepts of the MOOC by asking questions to the learners and providing examples. JavaPAL, a voice-based conversational agent for supporting learners on a MOOC on programming with Java offered on edX. This paper evaluates JavaPAL from different perspectives. First, the usability of JavaPAL is analyzed, obtaining a score of 74.41 according to a System Usability Scale (SUS). Second, learners? performance is compared when answering questions directly through JavaPAL and through the equivalent web interface on edX, getting similar results in terms of performance. Finally, interviews with JavaPAL users reveal that this conversational agent can be helpful as a complementary tool for the MOOC due to its portability and flexibility compared to accessing the MOOC contents through the web interface.

Massive open online courses (MOOCs) pose a challenge for instructors when trying to provide personalised support to learners, due to large numbers of registered participants. Conversational agents can be of help to support learners when working with MOOCs. This article presents an adaptive learning module for JavaPAL, a conversational agent that complements a MOOC on Java programming, helping learners review the key concepts of the MOOC. This adaptive learning module adapts the difficulty of the questions provided to learners considering their level of knowledge using item response theory (IRT) and also provides recommendations of video fragments extracted from the MOOC for when learners fail questions. The adaptive learning module for JavaPAL has been evaluated showing good usability and learnability through the system usability scale (SUS), reasonably suitable video fragments recommendations for learners, and useful visualisations generated as part of the IRT-based adaptation of questions for instructors to better understand what is happening in the course, to design exams, and to redesign the course content.