A Heuristic Method for Automatic Gaze Detection in Constrained Multi- Modal Dialogue Corpora

McLaren, Lorcan; Koutsombogera, Maria; Vogel, Carl

doi:10.1109/coginfocom50765.2020.9237883

Cited by 2 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For eye contact, we compared our used DeepEC approach and the benchmark, which is heuristic [ 47 ]. Table 12 represents the classification metrics for the DeepEC approach overall.…”

Section: Experimental Work and Analysismentioning

confidence: 99%

Real-Time Robotic Presentation Skill Scoring Using Multi-Model Analysis and Fuzzy Delphi–Analytic Hierarchy Process

Alshammari,

Abd Rahman,

Arshad

et al. 2023

Sensors

View full text Add to dashboard Cite

Existing methods for scoring student presentations predominantly rely on computer-based implementations and do not incorporate a robotic multi-classification model. This limitation can result in potential misclassification issues as these approaches lack active feature learning capabilities due to fixed camera positions. Moreover, these scoring methods often solely focus on facial expressions and neglect other crucial factors, such as eye contact, hand gestures and body movements, thereby leading to potential biases or inaccuracies in scoring. To address these limitations, this study introduces Robotics-based Presentation Skill Scoring (RPSS), which employs a multi-model analysis. RPSS captures and analyses four key presentation parameters in real time, namely facial expressions, eye contact, hand gestures and body movements, and applies the fuzzy Delphi method for criteria selection and the analytic hierarchy process for weighting, thereby enabling decision makers or managers to assign varying weights to each criterion based on its relative importance. RPSS identifies five academic facial expressions and evaluates eye contact to achieve a comprehensive assessment and enhance its scoring accuracy. Specific sub-models are employed for each presentation parameter, namely EfficientNet for facial emotions, DeepEC for eye contact and an integrated Kalman and heuristic approach for hand and body movements. The scores are determined based on predefined rules. RPSS is implemented on a robot, and the results highlight its practical applicability. Each sub-model is rigorously evaluated offline and compared against benchmarks for selection. Real-world evaluations are also conducted by incorporating a novel active learning approach to improve performance by leveraging the robot’s mobility. In a comparative evaluation with human tutors, RPSS achieves a remarkable average agreement of 99%, showcasing its effectiveness in assessing students’ presentation skills.

show abstract

“…For eye contact, we compared our used DeepEC approach and the benchmark, which is heuristic [ 47 ]. Table 12 represents the classification metrics for the DeepEC approach overall.…”

Section: Experimental Work and Analysismentioning

confidence: 99%

Real-Time Robotic Presentation Skill Scoring Using Multi-Model Analysis and Fuzzy Delphi–Analytic Hierarchy Process

Alshammari,

Abd Rahman,

Arshad

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

Gaze, Dominance and Dialogue Role in the MULTISIMO Corpus

McLaren

Koutsombogera

Vogel

2020

2020 11th IEEE International Conference on Cognitive Infocommunications (CogInfoCom)

Self Cite

View full text Add to dashboard Cite

Analysis of human interactions benefits from understanding patterns of gaze in dialogue. Using a multi-modal task oriented dialogue corpus, we evaluate relations between gaze behaviours, conversational dominance and dialogue role. Dominant players spend more time looking at their co-player than their less dominant fellows; the facilitator spends more time gazing at the dominant player than at the less dominant; dominant players spend longer averting their gaze than their less dominant fellows. When aligned with dialogue roles, gaze behaviour differences between players are demonstrated when occupying the role of speaker but not of listener. Less dominant speakers gaze longer at a facilitator, while effects on averted gaze and co-player directed gaze remain present for dominant speakers but not for dominant listeners. Finally, where dominance is not balanced among players, mutual gaze involving players only is less than mutual gaze involving facilitators.

show abstract

A Heuristic Method for Automatic Gaze Detection in Constrained Multi- Modal Dialogue Corpora

Cited by 2 publications

References 16 publications

Real-Time Robotic Presentation Skill Scoring Using Multi-Model Analysis and Fuzzy Delphi–Analytic Hierarchy Process

Real-Time Robotic Presentation Skill Scoring Using Multi-Model Analysis and Fuzzy Delphi–Analytic Hierarchy Process

Gaze, Dominance and Dialogue Role in the MULTISIMO Corpus

Contact Info

Product

Resources

About