Enhancing Autism Severity Classification: Integrating LSTM into CNNs for Multisite Meltdown Grading

Abstract

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by deficits in social interaction, verbal and non-verbal communication, and is often associated with cognitive and neurobehavioral challenges. Timely screening and diagnosis of ASD are crucial for early educational planning, treatment, family support, and timely medical intervention. Manual diagnostic methods are time-consuming and labor-intensive, underscoring the need for automated approaches to assist caretakers and parents. While various researchers have employed machine learning and deep learning techniques for ASD diagnosis, existing models often fall short in capturing the complexity of multisite meltdowns and fully leveraging the interdependence among these meltdowns for severity assessment in acquired facial images of children, hindering the development of a comprehensive grading system. This paper introduces a novel approach using a Long Short Term Memory (LSTM) integrated Convolution Neural Network (CNN) designed to identify multisite meltdowns and exploit their interdependence for severity assessment in ASD. The process begins with image pre-processing, involving discrete convolution filters for noise removal and contrast enhancement to improve image quality. The enhanced image then undergoes instance segmentation using the Segment Anything model to identify significant regions in the child's image. The segmented region is subjected to principal component analysis for feature extraction, and these features are utilized by the LSTM-integrated CNN for meltdown detection and severity classification. The model is trained using children's images extracted from videos, and testing is performed on videos captured during children's observations. Performance analysis reveals superior results, with a training accuracy of 88% and validation accuracy of 84%, outperforming conventional methods. This innovative approach not only enhances the efficiency of ASD diagnosis but also provides a more nuanced understanding of multisite meltdowns and their impact on severity, contributing to the development of a robust grading system.

Predicting meltdown situation in Autism and ADHD in real-time through camera using deep learning algorithm.

“…To find the severity of multisite meltdown in children suffering from ASD, LSTM integrated Convolution Neural Network was used. In this study to improve the image quality, "Segment anything model for segmentation" and PCA for feature extraction were applied [18], to isolate and pin point certain regions containing specific meltdown related feature within a child's image .The model showed a promising result but more diverse dataset was needed for a real-time application. For recognising micro-expressions of complex emotion in autistic children during meltdown S.K.…”