Classroom Video Assessment and Retrieval via Multiple Instance Learning

Abstract

We propose a multiple instance learning approach to contentbased retrieval of classroom video for the purpose of supporting human assessing the learning environment. The key element of our approach is a mapping between the semantic concepts of the assessment system and features of the video that can be measured using techniques from the fields of computer vision and speech analysis. We report on a formative experiment in content-based video retrieval involving trained experts in the Classroom Assessment Scoring System, a widely used framework for assessment and improvement of learning environments. The results of this experiment suggest that our approach has potential application to productivity enhancement in assessment and to broader retrieval tasks.

Exploring Semi-Supervised Learning for Audio-Based Automated Classroom Observations

“…Multimodal methods typically result in better performance since the different modalities are used in conjunction with each other resulting in a richer representation of the data giving the model more information (Summaira et al, 2021). One such system that utilized multimodal data was CLEVER (Classroom Evaluation and Video Retrieval) (Qiao & Beling, 2011) that used audio and video data from classrooms and classified them based on the CLASS protocol. CLEVER used video and audio metrics to bridge the gap between semantic assessment concepts to make them quantifiable and measurable.…”

Toward Automated Classroom Observation: Predicting Positive and Negative Climate

“…Some researchers have investigated how to identify teachers' behaviors and pedagogical strategies [13], [16], [30], [31]. Finally, during the past few years, a few projects have also emerged (including ours) that analyze the dynamics of an entire classroom, either as the collection of individual students [14] or an aggregate measure of many interacting participants [15], [32]. This can provide the raw data for teacher dashboards and also facilitate automated classroom observation coding.…”

“…Due to its popularity in educational research, recently some computational researchers have developed methods to automate aspects of the Classroom Assessment Scoring System (CLASS). The earliest work in this vein was by Qiao and Beling [32], who developed a computer vision system, optimized within a multipleinstance learning framework, to estimate which 3-minute clips of classroom videos were most relevant for CLASS coders to code manually. However, their system does not actually predict the CLASS scores themselves.…”