How Design Science Research Helps Improve Learning Efficiency in Online Conversations

Abstract

Abstract:In this design science research paper, we report on our constructing and evaluating an attention-guidance system that we integrated into a computer-supported collaborative learning system. Drawing on social constructivist literature, our proposed design focuses on attracting, retaining, and, if necessary, reacquiring users' attention on task-relevant information in online collaborative literature processing. The investigation involved an experiment across two sections of students in a human-computer interaction course. Results show that the new design allowed users to consistently reflect and evaluate the content of a text as they capitalized on one another's reasoning to resolve misconceptions. Moreover, we found that the new system increased users' perceptions of learning. However, the difference in knowledge gain scores was marginally significant and represented a medium effect size. Interestingly, we found that the attention-guidance system supported more efficient learning. Finally, we discovered that task-oriented reading of text, revisions of incomplete or incorrect ideas, and perceptions of learning mediated the relationship between software system and learning efficiency. We discuss the theoretical and practical implications.

Effects of recommendations on message quality and community formation in online conversations

“…Prior research has investigated the advantages of such systems. For example, Eryilmaz et al ( 2018 ) summarized the benefits of annotations on various learning activities such as (1) selecting relevant information, (2) articulating ideas by relating academic documents (e.g., journal articles) to prior understanding, (3) identifying comprehension gaps, and (4) revising incomplete and incorrect ideas. Chen et al ( 2020 ) noted that anchored or annotation-based discussion environments improve students’ reading comprehension, meta-cognitive skills, and learning outcomes via the immediacy of accessing information from text.…”

Integrating Learning Analytics to Measure Message Quality in Large Online Conversations

“…Our CSCL environment is the modular and flexible anchored AOD system developed in Eryilmaz et al [30] and extended by the addition of a recommender system in Eryilmaz et al [11]. When compared to AODs in existing learning management systems, this environment binds the instructional material and its related discussion in a single window to prevent students from toggling back and forth across multiple windows.…”

“…Prior research demonstrates that the abovementioned tight coupling facilitates a close spatial proximity between an instructional material and its associated discussion, which increases the communicative efficiency of AODs [30]. Moreover, this increase in communicative efficiency allows this system to produce a larger number of messages than AODs in existing learning management systems.…”

Development of a Reading Material Recommender System Based On Design Science Research Approach

“…We incorporated our instantiation artifact into a modular, flexible, and extensible anchored discussion system first developed by Eryilmaz et al [30]. In this CSCL environment, discussion threads are anchored to numbered and highlighted passages within the text to contextualize students' ideas.…”

“…Aiming to isolate the effects of the recommendation functionality, we implemented a control version of the anchored discussion system developed in [30]. This control system highlights both the annotated text and the pertinent discussion thread when either element is under the cursor.…”