Facial Recognition in Collaborative Learning Videos

Tran, Phuong Thi Minh; Pattichis, Marios S.; Celedón‐Pattichis, Sylvia; LópezLeiva, Carlos

doi:10.1007/978-3-030-89131-2_23

Cited by 6 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we note that ArcFace has been tested on a large-number of camera-facing image datasets and a variety of loss function models. We have summarized our modified system in a conference paper in [11]. For completeness, we will provide a summary of our methodology adopted from [11] in our methods section.…”

Section: ) Face Recognitionmentioning

confidence: 99%

“…We have summarized our modified system in a conference paper in [11]. For completeness, we will provide a summary of our methodology adopted from [11] in our methods section.…”

Section: ) Face Recognitionmentioning

confidence: 99%

“…We adopt the face recognition method previously described as a conference paper in [11]. We use the Insight-Face [23] system to recognize faces.…”

Section: Face Recognitionmentioning

confidence: 99%

“…We combine sparse sampling and K-means clustering to compute face prototypes as given in Algorithm 1 (also see [11]). Sample an image every 30 frames of video 3:…”

Section: Face Recognitionmentioning

confidence: 99%

“…We note that we presented preliminary results on group detection in conference publications: [7], [8], [9], [10], and video face recognition in [11]. While we review these earlier methods for completeness, we note that the current paper describes training and testing on the complete system over much larger datasets.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Long-Term Human Participation Assessment in Collaborative Learning Environments Using Dynamic Scene Analysis

Shi,

Tran,

Celedón-Pattichis

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

The paper develops datasets and methods to assess student participation in real-life collaborative learning environments. In collaborative learning environments, students are organized into small groups where they are free to interact within their group. Thus, students can move around freely causing issues with strong pose variation, move out and re-enter the camera scene, or face away from the camera. We formulate the problem of assessing student participation into two subproblems: (i) student group detection against strong background interference from other groups, and (ii) dynamic participant tracking within the group. A massive independent testing dataset of 12,518,250 student label instances, of total duration of 21 hours and 22 minutes of real-life videos, is used for evaluating the performance of our proposed method for student group detection. The proposed method of using multiple image representations is shown to perform equally or better than YOLO on all video instances. Over the entire dataset, the proposed method achieved an F1 score of 0.85 compared to 0.80 for YOLO. Following student group detection, the paper presents the development of a dynamic participant tracking system for assessing student group participation through long video sessions. The proposed dynamic participant tracking system is shown to perform exceptionally well, missing a student in just one out of 35 testing videos. In comparison, a state of the art method fails to track students in 14 out of the 35 testing videos. The proposed method achieves 82.3% accuracy on an independent set of long, real-life collaborative videos.INDEX TERMS Human participation assessment, Dynamic participant tracking, Occlusion detection.

show abstract

Section: ) Face Recognitionmentioning

confidence: 99%

“…We have summarized our modified system in a conference paper in [11]. For completeness, we will provide a summary of our methodology adopted from [11] in our methods section.…”

Section: ) Face Recognitionmentioning

confidence: 99%

“…We adopt the face recognition method previously described as a conference paper in [11]. We use the Insight-Face [23] system to recognize faces.…”

Section: Face Recognitionmentioning

confidence: 99%

“…We combine sparse sampling and K-means clustering to compute face prototypes as given in Algorithm 1 (also see [11]). Sample an image every 30 frames of video 3:…”

Section: Face Recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Long-Term Human Participation Assessment in Collaborative Learning Environments Using Dynamic Scene Analysis

Shi,

Tran,

Celedón-Pattichis

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

Fast Hand Detection in Collaborative Learning Environments

Teeparthi

Jatla

Pattichis

et al. 2021

Computer Analysis of Images and Patterns

Self Cite

View full text Add to dashboard Cite

Human activity recognition in videos is a challenging problem that has drawn a lot of interest, particularly when the goal requires the analysis of a large video database.The Advancing Out-of-School Learning in Mathematics and Engineering (AOLME) project provides a collaborative learning environment for middle school students to explore mathematics, computer science, and engineering by processing digital images and videos. As part of this project, around 2200 hours of video data were collected for analysis. This data was collected to understand how children learn in situations involving mathematical and programming challenges so as to recognize best teaching practices that support broadening participation of underrepresented students in STEM fields. Because of the size of the dataset, it is hard to analyze all the videos of the dataset manually. Thus, there is a huge need for reliable computerbased methods that can detect activities of interest.My thesis is focused on the development of accurate methods for detecting and tracking objects in collaborative learning environments in long videos (> 1 hour). v Long-term object detection and tracking face fundamental challenges due to occlusion, illumination variations, and pose variations.For collaborative learning groups, the thesis contributes robust methods for computer keyboard detection, tracking, and student hand detection. For hand detection, the thesis integrates object detection with clustering and time-projections for accurate, long-term assessment of student participation. The hand detection method was integrated into a writing detection system and can also be used for later research on recognizing student gestures.All the models are validated on videos from 7 different sessions, ranging from 45 minutes to 90 minutes. The keyboard detector achieved a very high average precision (AP) of 92% at 0.5 intersection over union (IoU). Furthermore, a combined system of the detector with a fast tracker KCF (159fps) was developed so that the algorithm runs significantly faster without sacrificing accuracy. For a video of 23 minutes having resolution 858 × 480 @ 30 fps, the detection alone runs at 4.7×the real-time, and the combined algorithm runs at 21×the real-time for an average IoU of 0.84 and 0.82, respectively. The hand detector achieved average precision (AP) of 72% at 0.5 intersection over union (IoU). The detection results were improved to 81% using optimal data augmentation parameters. The hand detector runs at 4.7×the real-time with AP of 81% at 0.5 intersection over union. The hand detection method was integrated with projections and clustering for accurate proposal generation. This approach reduced the number of false-positive hand detections by 80%. The overall hand detection system runs at 4×the real-time, capturing all the activity regions of the current collaborative group. vi Contents List of Figures x List of Tables xiii Glossary xvAR Average Recall.

show abstract