Guided Weak Supervision for Action Recognition with Scarce Data to Assess Skills of Children with Autism

Pandey, Prashant; Ap, Prathosh; Kohli, Manu; Pritchard, Josh

doi:10.1609/aaai.v34i01.5383

Cited by 22 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future work, we will establish a strategy to incorporate multiple annotation distributions from all annotators, and investigate more visual behavior indicators including verbal and non-verbal clues for emotion [38] and engagement [4,5,27,28] assessment. We also would like to leverage publicly available large-scale benchmark datasets to reduce human annotation cost via transfer learning [42]. Since the current framework analyzes video footage offline, we wish to build a real-time feedback and assessment system [6].…”

Section: Discussionmentioning

confidence: 99%

Improving the Movement Synchrony Estimation with Action Quality Assessment in Children Play Therapy

Bhat

Barmaki

2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Movement synchrony refers to the dynamic temporal connection between the motions of interacting people. The automatic measurement of movement synchrony is worth studying for social behavior analysis applications, for instance, in play therapy of children in the autism spectrum. Existing approaches based on motion energy analysis are strongly reliant on the region of interest, and thus limit the interaction between individuals, especially for highly engaging activities like play therapy. Inspired by action quality assessment, a task to assess how well an action has been performed, in this paper, we propose an end-to-end deep learning method to integrate the following major tasks: (1) the automatic assessment of children's performance in play therapy, and (2) the automatic estimation of movement synchrony between children and therapists, facilitated by an auxiliary task of intervention activity recognition. This multi-task paradigm generally improves the performance of our model across all tasks. Furthermore, when annotations are subjective, the typical exclusive annotation strategy may reduce tagging quality. As a result, we explored applying distribution learning to mitigate human bias in movement synchrony estimation. We allowed the second and third labels for each instance, namely the uncertainty-preserved annotation approach. We tested our method on Play Therapy 13 (PT13), a dataset collected from video recordings of play therapy interventions. The findings of the experiments indicated that our framework can accurately quantify movement synchronization and assess the quality of children's actions in play therapy. Moreover, the uncertainty-preserved annotation approach produced a comparable outcome to standard methods at a far reduced cost, demonstrating its efficacy in mitigating biases.

show abstract

Section: Discussionmentioning

confidence: 99%

Improving the Movement Synchrony Estimation with Action Quality Assessment in Children Play Therapy

Bhat

Barmaki

2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

show abstract

“…Pandey et al [12] devised the Guided Weak Supervision (GWS) method, in which each class of the target data of the small dataset is matched to the source data that posses large amount of annotated data, by using posterior likelihood maximization. Rajagopalan et al [13] have published a video based dataset for stimming behaviour analysis of a child.…”

Section: Related Workmentioning

confidence: 99%

Detecting A Child’s Stimming Behaviours for Autism Spectrum Disorder Diagnosis using Rgbpose-Slowfast Network

Pandian

Rajagopalan

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Autism Spectrum Disoder (ASD) is a neurodevelopmental disorder characterized by (a) persistent deficits in social communication and interaction, and (b) presence of restrictive, repetitive patterns of behaviours, interests or activities. The stereotyped repetitive behaviours are also referred to as stimming behaviours. We propose a deep learning based approach to automatically predict a child's stimming behaviours from videos recorded in unconstrained conditions. The child's region in the video is tracked and its skeletal joints are derived using the pose estimator. The heatmap representation of skeletal joints and the raw video signals are used as inputs to the two pathways of the RGBPose-SlowFast deep network to model stimming behaviours. The proposed model is evaluated using the publicly available Self-Stimulatory Behaviour Dataset (SSBD) of stimming behaviours. The generalization ability of the model is validated using the Autism dataset containing child's motor actions. Our experiments demonstrate state-of-the-art results on both datasets.

show abstract

“…Artificial Intelligence (A.I.) based innovations have fasttracked ASD diagnostics [31], [32], increased clinician capacity, and improved access to early intervention programs [26]. The adoption of these technologies has surged during the COVID-19 pandemic [33].…”

Section: Introductionmentioning

confidence: 99%

The Role of Intelligent Technologies in Early Detection of Autism Spectrum Disorder (ASD): A Scoping Review

2022

Self Cite

View full text Add to dashboard Cite

Background: Two-year delay is reported between the first developmental concern raised by the parents and the diagnosis of ASD (Autism Spectrum Disorder), delaying the start of early intervention programs most beneficial within the first three years. Aim: Evaluate the role of technology in ASD detection by answering four research questions analyzing 1) evolution of technology, 2) use of various bio-behavioral data sources, 3) demographic categories, databases, controls, comparators, and assessment instruments, and 4) data collection, processing, and outcomes of the technology-based methods in ASD detection. Methods: Scoping review included behavioral-based ASD screening and diagnostic studies, published between 1st January 2011 to 31st December 2021 in PUBMED, SCOPUS, and IEEE Xplore databases for children under six years. The studies were assessed using the Critical Appraisal Skills Programm (CASP) and the PRISMA scoping review checklist (PRISMA-ScR). Results: The shortlisted 35 studies were categorized into seven bio-behavioral categories. The review suggested extensive use of machine learning (ML) and Deep Learning (DL) technologies with multimodal structured and unstructured data to detect infants at risk of ASD and Other developmental delays (ODD) as early as 9 to 12 months. However, the review reported various internal and external validity threats. Conclusion: Technology can significantly improve the current ASD detection process. The validation and adoption of technology can be fast-tracked by 1) designing robust study protocols, 2) executing multicultural field trials, 3) standardizing datasets, data quality and feature engineering methods, 4) recruiting statistically significant participants from ASD, typically developing (TD) and other developmental disorders (ODD) groups to ensure technological generalization, validation, and adoption outside laboratory settings.

show abstract

Guided Weak Supervision for Action Recognition with Scarce Data to Assess Skills of Children with Autism

Cited by 22 publications

References 17 publications

Improving the Movement Synchrony Estimation with Action Quality Assessment in Children Play Therapy

Improving the Movement Synchrony Estimation with Action Quality Assessment in Children Play Therapy

Detecting A Child’s Stimming Behaviours for Autism Spectrum Disorder Diagnosis using Rgbpose-Slowfast Network

The Role of Intelligent Technologies in Early Detection of Autism Spectrum Disorder (ASD): A Scoping Review

Contact Info

Product

Resources

About