Automated Detection of Facial Expressions during Computer-Assisted Instruction in Individuals on the Autism Spectrum

Ahmed, Alex A.; Goodwin, Matthew S.

doi:10.1145/3025453.3025472

Cited by 7 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Computer vision has also been used to predict engagement and learning performance. For example, Ahmed and Goodwin 53 analysed facial expressions from video recordings obtained when kids interacted with a computer-assisted instruction programme. Their results showed that emotional and behavioural engagement can be quantified automatically using computer vision analysis.…”

Section: Facial Expression/emotionmentioning

confidence: 99%

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

et al. 2020

View full text Add to dashboard Cite

The current state of computer vision methods applied to autism spectrum disorder (ASD) research has not been well established. Increasing evidence suggests that computer vision techniques have a strong impact on autism research. The primary objective of this systematic review is to examine how computer vision analysis has been useful in ASD diagnosis, therapy and autism research in general. A systematic review of publications indexed on PubMed, IEEE Xplore and ACM Digital Library was conducted from 2009 to 2019. Search terms included [‘autis*’ AND (‘computer vision’ OR ‘behavio* imaging’ OR ‘behavio* analysis’ OR ‘affective computing’)]. Results are reported according to PRISMA statement. A total of 94 studies are included in the analysis. Eligible papers are categorised based on the potential biological/behavioural markers quantified in each study. Then, different computer vision approaches that were employed in the included papers are described. Different publicly available datasets are also reviewed in order to rapidly familiarise researchers with datasets applicable to their field and to accelerate both new behavioural and technological work on autism research. Finally, future research directions are outlined. The findings in this review suggest that computer vision analysis is useful for the quantification of behavioural/biological markers which can further lead to a more objective analysis in autism research.

show abstract

Section: Facial Expression/emotionmentioning

confidence: 99%

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Scassellati et al (2012) found that social robots may be developed into useful tools for social skills and communication therapies, specifically by embedding social interaction into intrinsic reinforcers and motivators for children with ASD [50]. Ahmed et al have found that for children with autism, behavioral engagement (the proportion of time a participant has their face oriented to the computer screen) and emotional engagement (the activation of facial activation units) are strongly correlated with one another, but that emotional (not behavioral) engagement is a better predictor of test performance [1]. Our system takes a different approach to addressing this phenomenon by providing feedback reinforcing behavioral engagement for the child only when the child elicits emotional engagement with a conversational partner.…”

Section: Behavioral Therapy Systems For Asdmentioning

confidence: 99%

SuperpowerGlass

Washington

Voss

Kline

et al. 2017

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

We have developed a system for automatic facial expression recognition running on Google Glass, delivering real-time social cues to children with Autism Spectrum Disorder (ASD). The system includes multiple mechanisms to engage children and their parents, who administer this technology within the home. We completed an at-home design trial with 14 families that used the learning aid over a 3-month period. We found that children with ASD generally respond well to wearing the system at home and opt for the most expressive feedback choice. We further evaluated app usage, facial engagement, and model accuracy. We found that the device can act as a powerful training aid when used periodically in the home, that interactive video content from wearable therapy sessions should be augmented with sufficient context about the content to produce long-term engagement, and that the design of wearable systems for children with ASD should be heavily dependent on the functioning level of the child. We contribute general design implications for developing wearable aids used by children with ASD and other behavioral disorders as well as their parents during at-home parent-administered therapy sessions.

show abstract

“…The sample size of the included studies ranged from 2 to 35 children and/or youth, with a mean age up to 20.8 years (see Table 1 ). Of the 12 studies that reported on child and/or youth gender, 11 studies (92%) included male majority samples ( 33 , 34 , 38 – 41 , 45 , 46 , 49 , 50 , 53 ). The vast majority of the included studies focused on children and/or youth with autism spectrum disorder (ASD) ( n = 19/21; 91%) ( 33 , 34 , 37 – 53 ), followed by single instances of studies including children with Down syndrome (5%) ( 50 ), children with a visual impairment (5%) ( 35 ), and children with cerebral palsy (5%) ( 36 ).…”

Section: Resultsmentioning

confidence: 99%

“…None of the studies reported on family socio-economic status, family income, or child or youth ethnicity [Hispanic, non-Hispanic]. Only 1 study reported on child or youth race ( 33 ) and only 1 study reported on parental education level for one of the caregivers sampled ( 40 ).…”

Section: Resultsmentioning

confidence: 99%

“…These annotated observations were paired with data collected from facial, skeleton, or eye recognition tools ( n = 9/20; 45%) ( 33 , 37 , 40 – 42 , 44 , 45 , 50 , 52 ), sensors ( n = 6/20; 30%) ( 34 , 40 , 44 , 45 , 51 , 53 ), EEG ( n = 4/20; 20%) ( 34 , 36 , 39 , 46 ), via distance estimates ( n = 3/20; 15%) ( 47 – 49 ), and/or other tools ( n = 6/20; 30%) such as microphones or electrodes ( 34 , 38 , 40 , 43 – 45 ). To capture or predict participation, 18 of the 21 participation assessment approaches used multiple types of AI (86%) ( 34 – 50 , 52 ).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Capturing and Operationalizing Participation in Pediatric Re/Habilitation Research Using Artificial Intelligence: A Scoping Review

Kaelin

Valizadeh

Salgado

et al. 2022

Front. Rehabilit. Sci.

View full text Add to dashboard Cite

BackgroundThere is increased interest in using artificial intelligence (AI) to provide participation-focused pediatric re/habilitation. Existing reviews on the use of AI in participation-focused pediatric re/habilitation focus on interventions and do not screen articles based on their definition of participation. AI-based assessments may help reduce provider burden and can support operationalization of the construct under investigation. To extend knowledge of the landscape on AI use in participation-focused pediatric re/habilitation, a scoping review on AI-based participation-focused assessments is needed.ObjectiveTo understand how the construct of participation is captured and operationalized in pediatric re/habilitation using AI.MethodsWe conducted a scoping review of literature published in Pubmed, PsycInfo, ERIC, CINAHL, IEEE Xplore, ACM Digital Library, ProQuest Dissertation and Theses, ACL Anthology, AAAI Digital Library, and Google Scholar. Documents were screened by 2–3 independent researchers following a systematic procedure and using the following inclusion criteria: (1) focuses on capturing participation using AI; (2) includes data on children and/or youth with a congenital or acquired disability; and (3) published in English. Data from included studies were extracted [e.g., demographics, type(s) of AI used], summarized, and sorted into categories of participation-related constructs.ResultsTwenty one out of 3,406 documents were included. Included assessment approaches mainly captured participation through annotated observations (n = 20; 95%), were administered in person (n = 17; 81%), and applied machine learning (n = 20; 95%) and computer vision (n = 13; 62%). None integrated the child or youth perspective and only one included the caregiver perspective. All assessment approaches captured behavioral involvement, and none captured emotional or cognitive involvement or attendance. Additionally, 24% (n = 5) of the assessment approaches captured participation-related constructs like activity competencies and 57% (n = 12) captured aspects not included in contemporary frameworks of participation.ConclusionsMain gaps for future research include lack of: (1) research reporting on common demographic factors and including samples representing the population of children and youth with a congenital or acquired disability; (2) AI-based participation assessment approaches integrating the child or youth perspective; (3) remotely administered AI-based assessment approaches capturing both child or youth attendance and involvement; and (4) AI-based assessment approaches aligning with contemporary definitions of participation.

show abstract

Automated Detection of Facial Expressions during Computer-Assisted Instruction in Individuals on the Autism Spectrum

Cited by 7 publications

References 26 publications

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

SuperpowerGlass

Capturing and Operationalizing Participation in Pediatric Re/Habilitation Research Using Artificial Intelligence: A Scoping Review

Contact Info

Product

Resources

About