Crowdsourcing facial responses to online videos: Extended abstract

Abstract

Abstract-Traditional observational research methods required an experimenter's presence in order to record videos of participants, and limited the scalability of data collection to typically less than a few hundred people in a single location. In order to make a significant leap forward in affective expression data collection and the insights based on it, our work has created and validated a novel framework for collecting and analyzing facial responses over the Internet. The first experiment using this framework enabled 3,268 trackable face videos to be collected and analyzed in under two months. Each participant viewed one or more commercials while their facial response was recorded and analyzed. Our data showed significantly different intensity and dynamics patterns of smile responses between subgroups who reported liking the commercials versus those who did not. Since this framework appeared in 2011, we have collected over three million videos of facial responses in over 75 countries using this same methodology, enabling facial analytics to become significantly more accurate and validated across five continents. Many new insights have been discovered based on crowdsourced facial data, enabling Internet-based measurement of facial responses to become reliable and proven. We are now able to provide large-scale evidence for gender, cultural and age differences in behaviors. Today such methods are used as part of standard practice in industry for copy-testing advertisements and are increasingly used for online media evaluations, distance learning, and mobile applications.

Flat vs. Expressive Storytelling: Young Children’s Learning and Retention of a Social Robot’s Narrative

“…Although no data has been published yet specifically comparing the performance of the software on adults vs. children, FACS coding is generally the same for adults and for children and has been used with children as young as 2 years (e.g., Camras et al, 2006 ; LoBue and Thrasher, 2015 ; also see Ekman and Rosenberg, 1997 ). Furthermore, this software has been trained and tested on tens of thousands of manually coded images of faces from around the world (McDuff et al, 2013 , 2015 ; Senechal et al, 2015 ).…”

In the Eye of the Beholder: Changes in Perceived Emotion Expression after Smile Reanimation

“…10 Freely available, facial recognition and emotion detection software has been developed from 5 million videos of normal subjects from 75 countries, resulting in over 2 billion facial frames in which Facial Action Coding System action units were expertly recognized and labeled. [11][12][13][14][15] These data have enabled automatic analysis of the phenomenon known as emotionality, the observable behavioral and physiologic component of emotion. 16 One must recognize that facial movement analysis alone, without contextual information, is unable to determine the true or felt emotional state of an individual.…”

Towards the Prediction of the Quality of Experience from Facial Expression and Gaze Direction

“…Many studies can be found regarding facial expression recognition, (e.g., [12]), but these are just focused on the association of facial expressions extracted from face images to a specific emotion (e.g., happiness, sadness, anger). As a practical example, [13] presents a crowdsourcing webbased framework called Affectiva 2 , which allows to collect and analyze facial expressions of video viewers to provide unobtrusive evaluation of facial responses to media content without relying on self-report ratings. However, the main objective is to determine the viewer's emotional engagement whereas the perceived quality is not considered.…”