Student and Instructor Experience Using Collaborative Annotation via Perusall in Upper Year and Graduate Courses

Abstract

Perusall is a collaborative annotation platform designed for pre-readings in a flipped classroom, but can also be used for stand-alone, asynchronous reading discussion components of courses. We examine the use of Perusall as a social constructivist learning tool in two upper year/graduate courses in Mechanical Engineering. Perusall was used to replace in-class discussion of readings during the shift to online teaching. Data was collected from student surveys and from the student and instructor annotations themselves. Annotations were coded for content, and examined for factors such as upvoting. We found substantial engagement from students, with collaborative annotation providing opportunities for: correction of misunderstanding; linking concepts from the course and between readings; discussing larger issues around research and research writing; sharing background information among peers; and critically analyzing the readings. Students reported deeper learning than in typical in-class discussions of readings; however, they also noted that annotation required much more time. Overall, collaborative annotation appears to be an effective method for course reading discussion.

High-tech versus low-tech: Comparing approaches to increase college student reading compliance and engagement.

“…It also appears that new, more technologically advanced social annotation tools can be as effective as classic low-tech options like reading responses for improving reading compliance in courses. However, prior research (e.g., D’Entremont & Eyking, 2021; Liberatore, 2017) and qualitative responses in the present study converge to show that no compliance tool solves all problems. Clever students could earn full credit for auto-graded annotations that are made by simply skimming a text or other students’ annotations; reading responses could be written after simply reading a summary at the end of a textbook chapter; and even reading quizzes could reward or punish students not based on their reading compliance, but on their ability to quickly and easily grasp main concepts from a reading (thus, unintentionally rewarding students for ability when effort, instead, is what’s desired).…”

“…This failure to find differences also does not appear to be simply a result of low statistical power in spite of large effect sizes; means across groups were nearly identical. I expected Perusall to have some advantages in these areas due to its interactive nature (e.g., D’Entremont & Eyking, 2021; Lazzara & Clinton-Lisell, 2022). However, since the course examined here was conducted in a face-to-face modality, it could be that enough interaction was taking place within the class setting that additional interaction outside of class was not a salient learning need for students in the reading response group (cf.…”

“…More recent research has also demonstrated the need for instructors to avoid thinking of social annotation tools as simply a panacea for course preparation problems and be intentional about how they are implemented (e.g., Allred et al, 2020; Chan et al, 2017; Suhre et al, 2019). Some potential weaknesses in using these tools can include students interacting with one another in a superficial way or just making comments to meet a quota rather than deeply interacting with the text (Allred et al, 2020; D’Entremont & Eyking, 2021; Liberatore, 2017). Additionally, students may avoid using these tools if instructors fail to provide adequate training (Suhre et al, 2019) or if use is optional and there are other more well-known tools that can adequately meet students’ needs (Chan et al, 2017).…”

“…Instructors also have the ability to override grades. While some question the validity of the annotation grading (e.g., valuing length of the comment over quality, D’Entremont & Eyking, 2021) others have found that the machine grading was mostly consistent with instructor evaluations of annotations (Cecchinato & Foschi, 2020) even if length appeared to be a factor (Liberatore, 2017). Cecchinato and Foschi (2020) found that in cases where scoring between Perusall and the instructor was not consistent, Perusall was much more likely to give students a slightly higher score versus lower score than the instructor.…”

“…Researchers who have examined the effectiveness of using Perusall for improving reading compliance and engagement have generally found it to be a helpful tool. Perusall can be used in an online course to effectively facilitate building scholarly community among both undergraduate (D'Entremont & Eyking, 2021) and graduate students (Adams & Wilson, 2020). Students appear to respond positively to social annotation via Perusall and even exceed instructor expectations (Lazzara & Clinton-Lisell, 2022).…”