Automatic Recognition of Posed Facial Expression of Emotion in Individuals with Autism Spectrum Disorder

Manfredonia, Joseph; Bangerter, Abigail; Manyakov, Nikolay V.; Ness, Seth; Lewin, David; Skalkin, Andrew; Boice, Matthew; Goodwin, Matthew S.; Dawson, Geraldine; Hendren, Robert L.; Leventhal, Bennett L.; Shic, Frederick; Pandina, Gahan

doi:10.1007/s10803-018-3757-9

Cited by 65 publications

(48 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Also, the more time ASD participants spent looking at the eye region of images of faces in the direct and averted gaze paradigm, the better they scored on the ABI social communications domain (Figure 5C). Detailed presentations of results on FACET and eye-tracking are available elsewhere (Manfredonia et al, 2018; Manyakov et al, 2018) and manuscripts of additional results from these and other sensors are in preparation.…”

Section: Resultsmentioning

confidence: 99%

“…In this study, we report on the practicality of combining and scaling multiple methodologies including biosensors and clinician and caregiver reporting to measure outcomes in a multisite study of children, adolescents, and adults with ASD. Recently, results from specific features of My JAKE and JAKE sense have been reported (Bangerter et al, 2017, In press; Manfredonia et al, 2018; Manyakov et al, 2018; Jagannatha et al, In press; Sargsyan et al, In press).…”

Section: Discussionmentioning

confidence: 99%

“…System reliability, quality, user feedback, and representative examples of results from the various components of JAKE are described herein as well as participant demographics and safety. Detailed methodology and results of the ABI, other components of My JAKE, and the various sensors and tasks are published elsewhere (Bangerter et al, 2017, In press; Manfredonia et al, 2018; Manyakov et al, 2018; Jagannatha et al, In press) or are in preparation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Observational Study With the Janssen Autism Knowledge Engine (JAKE®) in Individuals With Autism Spectrum Disorder

et al. 2019

Self Cite

View full text Add to dashboard Cite

Objective: The Janssen Autism Knowledge Engine (JAKE®) is a clinical research outcomes assessment system developed to more sensitively measure treatment outcomes and identify subpopulations in autism spectrum disorder (ASD). Here we describe JAKE and present results from its digital phenotyping (My JAKE) and biosensor (JAKE Sense) components. Methods: An observational, non-interventional, prospective study of JAKE in children and adults with ASD was conducted at nine sites in the United States. Feedback on JAKE usability was obtained from caregivers. JAKE Sense included electroencephalography, eye tracking, electrocardiography, electrodermal activity, facial affect analysis, and actigraphy. Caregivers of individuals with ASD reported behaviors using My JAKE. Results from My JAKE and JAKE Sense were compared to traditional ASD symptom measures. Results: Individuals with ASD ( N = 144) and a cohort of typically developing (TD) individuals ( N = 41) participated in JAKE Sense. Most caregivers reported that overall use and utility of My JAKE was “easy” (69%, 74/108) or “very easy” (74%, 80/108). My JAKE could detect differences in ASD symptoms as measured by traditional methods. The majority of biosensors included in JAKE Sense captured sizable amounts of quality data (i.e., 93–100% of eye tracker, facial affect analysis, and electrocardiogram data was of good quality), demonstrated differences between TD and ASD individuals, and correlated with ASD symptom scales. No significant safety events were reported. Conclusions: My JAKE was viewed as easy or very easy to use by caregivers participating in research outside of a clinical study. My JAKE sensitively measured a broad range of ASD symptoms. JAKE Sense biosensors were well-tolerated. JAKE functioned well when used at clinical sites previously inexperienced with some of the technologies. Lessons from the study will optimize JAKE for use in clinical trials to assess ASD interventions. Additionally, because biosensors were able to detect features differentiating TD and ASD individuals, and also were correlated with standardized symptom scales, these measures could be explored as potential biomarkers for ASD and as endpoints in future clinical studies. Clinical Trial Registration: https://clinicaltrials.gov/ct2/show/NCT02668991 identifier: NCT02668991

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Observational Study With the Janssen Autism Knowledge Engine (JAKE®) in Individuals With Autism Spectrum Disorder

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…This biological motion task was part of a large, observational, multi-center study conducted from 06 July 2015 to 14 October 2016 at nine study sites in the US (ClinicalTrials. gov, NCT02668991) and consisted of passive viewing tasks 12.0 (6-54) 11.5 (6-63) P-value 0.69 ADOS-2 total score, mean (SD, range) 7.6 (1.7, 4-10) -KBIT-2 IQ composite score, mean (SD, range) 98.5 (20.0, 60-136) - (Bangerter et al 2020a(Bangerter et al , 2020bJagannatha et al 2019;Manfredonia et al 2018;Manyakov et al 2018;Ness et al 2019;Sargsyan et al 2019). In this study, both groups of participants completed the same set of biosensor tasks.…”

Section: Biological Motionmentioning

confidence: 99%

Visual Preference for Biological Motion in Children and Adults with Autism Spectrum Disorder: An Eye-Tracking Study

Kaliukhovich

Manyakov

Bangerter

et al. 2020

J Autism Dev Disord

Self Cite

View full text Add to dashboard Cite

Participants with autism spectrum disorder (ASD) (n = 121, mean [SD] age: 14.6 [8.0] years) and typically developing (TD) controls (n = 40, 16.4 [13.3] years) were presented with a series of videos representing biological motion on one side of a computer monitor screen and non-biological motion on the other, while their eye movements were recorded. As predicted, participants with ASD spent less overall time looking at presented stimuli than TD participants (P < 10–3) and showed less preference for biological motion (P < 10–5). Participants with ASD also had greater average latencies than TD participants of the first fixation on both biological (P < 0.01) and non-biological motion (P < 0.02). Findings suggest that individuals with ASD differ from TD individuals on multiple properties of eye movements and biological motion preference.

show abstract

“…One bottleneck in facial expression studies is the rigorous manual coding of emotional expression through facial affect coding. However, the advent of new computer vision software capable of automated facial expression analysis and subsequent reductions in analysis time has enabled researchers to obtain larger samples of individuals with ASD [20]. For example, the Autism and Beyond study utilized automatic coding of over 4000 video samples to establish differences in emotion expression in toddlers who had a high likelihood of future ASD diagnosis [21].…”

Section: Introductionmentioning

confidence: 99%

Automated recognition of spontaneous facial expression in individuals with autism spectrum disorder: parsing response variability

et al. 2020

Self Cite

View full text Add to dashboard Cite

Background: Reduction or differences in facial expression are a core diagnostic feature of autism spectrum disorder (ASD), yet evidence regarding the extent of this discrepancy is limited and inconsistent. Use of automated facial expression detection technology enables accurate and efficient tracking of facial expressions that has potential to identify individual response differences. Methods: Children and adults with ASD (N = 124) and typically developing (TD, N = 41) were shown short clips of "funny videos." Using automated facial analysis software, we investigated differences between ASD and TD groups and within the ASD group in evidence of facial action unit (AU) activation related to the expression of positive facial expression, in particular, a smile. Results: Individuals with ASD on average showed less evidence of facial AUs (AU12, AU6) relating to positive facial expression, compared to the TD group (p < .05, r = − 0.17). Using Gaussian mixture model for clustering, we identified two distinct distributions within the ASD group, which were then compared to the TD group. One subgroup (n = 35), termed "over-responsive," expressed more intense positive facial expressions in response to the videos than the TD group (p < .001, r = 0.31). The second subgroup (n = 89), ("under-responsive"), displayed fewer, less intense positive facial expressions in response to videos than the TD group (p < .001; r = − 0.36). The overresponsive subgroup differed from the under-responsive subgroup in age and caregiver-reported impulsivity (p < .05, r = 0.21). Reduced expression in the under-responsive, but not the over-responsive group, was related to caregiver-reported social withdrawal (p < .01, r = − 0.3). Limitations: This exploratory study does not account for multiple comparisons, and future work will have to ascertain the strength and reproducibility of all results. Reduced displays of positive facial expressions do not mean individuals with ASD do not experience positive emotions.

show abstract

Automatic Recognition of Posed Facial Expression of Emotion in Individuals with Autism Spectrum Disorder

Cited by 65 publications

References 57 publications

An Observational Study With the Janssen Autism Knowledge Engine (JAKE®) in Individuals With Autism Spectrum Disorder

An Observational Study With the Janssen Autism Knowledge Engine (JAKE®) in Individuals With Autism Spectrum Disorder

Visual Preference for Biological Motion in Children and Adults with Autism Spectrum Disorder: An Eye-Tracking Study

Automated recognition of spontaneous facial expression in individuals with autism spectrum disorder: parsing response variability

Contact Info

Product

Resources

About