Board 39: Designing Intelligent Review Forms for Peer Assessment: A Data-driven Approach

Abstract

Florida in the Department of Computer Science and Engineering. He received his M.S. in Computer Science from USF in May 2017 and a B.A. in Computer Science and a B.A. in Applied Mathematics from Franklin College in May 2015. His teaching and research interests include Data Mining, Natural Language Processing (sentiment analysis, text processing), Crowd sourcing, Cyberlearning, Software Engineering, and Data Structures. He is a musician (guitar and bass) and was a collegiate athlete (soccer and tennis).

Through the Looking Glass: Insights into Visualization Pedagogy through Sentiment Analysis of Peer Review Text

“…Rubrics are used by instructors in a variety of disciplines to provide feedback or to grade student products, e.g., writings, presentations, and portfolios [3]. Generally speaking, a rubric is a pedagogical tool that articulates the expectations for an assignment.…”

Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment

“…The analytical section of our review form was created using an iterative, data-driven approach [5]. Our seed growing algorithm began with a basic rubric and questions were added, modified, or removed after each semester through intelligent data combing: a process of selecting informationrich keywords and phrases, through human intelligence, for the purpose of correctly analyzing and summarizing student observations.…”

“…The analytic grader simply matched student responses with their assigned values and aggregated the score. The process was relatively straightforward and is detailed in a prior paper [5]. Ultimately, the sentiment score (with accompanying metrics) was weighted with the analytic score and provided to the instructor as a suggested final grade from the peer-review process.…”

“…Our work addresses both issues in a novel way by leveraging sentiment analysis in the peer-review process. By organizing our courses to include peer teaching and crowdsourced peer review (as in [5]), we obtain a crowd of reviewers large enough (30-40 on average, although we have found 20 to be sufficiently stable) to confidently assess a student's work and as a side effect stimulate learning in reviewing students [6]- [8].…”

Leveraging Peer Feedback to Improve Visualization Education

“…Student self-review comments were collected on 7 projects from 2018 only 7 . We analyzed them with a dictionary-based natural language processing algorithm [3] for matching positive and negative keywords to produce numerical feedback including overall sentiment (positive or negative) of the text, counts of parts of speech (i.e., noun, adjective, adverb), the average length of comments, etc. The algorithm includes an aspect extractor (similar to one developed by Google towards analyzing reviews of local services, such as restaurants and hotels [7]) that scans text in a sliding window and produces a list of important aspects (i.e., nouns), which are in close proximity to sentiment words (i.e., adjectives).…”