Wearable-Sensors-Based Platform for Gesture Recognition of Autism Spectrum Disorder Children Using Machine Learning Algorithms

Abstract

Autistic people face many challenges in various aspects of daily life such as social skills, repetitive behaviors, speech, and verbal communication. They feel hesitant to talk with others. The signs of autism vary from one individual to another, with a range from mild to severe. Autistic children use fewer communicative gestures compared with typically developing children (TD). With time, the parents may learn their gestures and understand what is occurring in their child’s mind. However, it is difficult for other people to understand their gestures. In this paper, we propose a wearable-sensors-based platform to recognize autistic gestures using various classification techniques. The proposed system defines, monitors, and classifies the gestures of the individuals. We propose using wearable sensors that transmit their data using a Bluetooth interface to a data acquisition and classification server. A dataset of 24 gestures is created by 10 autistic children performing each gesture about 10 times. Time- and frequency-domain features are extracted from the sensors’ data, which are classified using k-nearest neighbor (KNN), decision tree, neural network, and random forest models. The main objective of this work is to develop a wearable-sensor-based IoT platform for gesture recognition in children with autism spectrum disorder (ASD). We achieve an accuracy of about 91% with most of the classifiers using dataset cross-validation and leave-one-person-out cross-validation.

Automatic Decision-Making Style Recognition Method Using Kinect Technology

“…In addition, Heraz and Clynes (2018) identified subjects' emotions by measuring the coordinates, strength, and skin area of finger touch screens while operating their phones, and Chen et al (2019) proposed a method to identify decision-making styles using text, images, and videos (digital footprints) on Facebook. Siddiqui et al (2021) developed a wearable sensor-based platform to identify autism spectrum disorder (ASD) using machine learning. Asaduzzaman et al (2021) developed a system by using machine learning and data mining approach to predict the risk level of cervical and ovarian cancer in association to stress.…”

A Framework for Maternal Physical Activities and Health Monitoring Using Wearable Sensors

“…Table 2 shows the literature summary of sensors, physical activities, the window size with overlapping and non-overlapping for features extraction, and corresponding extracted features. We adapted most of the features from the works in [ 45 , 46 , 47 , 48 ].…”

Machine learning-based classification of the movements of children with profound or severe intellectual or multiple disabilities using environment data features

“…There is also evidence that eye-tracking technology is a potential satisfactory technology in supporting communication and psychosocial functioning of children with Rett syndrome who have speech and hand use impairments and severe motor apraxia [ 14 ]. Wearable-sensor-based platform systems consisting of gyroscopes, accelerometers, and global positioning system (GPS) have also been used to recognize the gesture movements of children with autism spectrum disorder (ASD) and communicate using several ML classification algorithms with mostly 91% accuracy rates [ 15 ].…”

“…Recent ML-based studies that aimed to classify or differentiate disorder sub-populations, distinguish behavioral phenotypes between and among disorders, and diagnosis, significantly targeted children with ASD, attention-deficit and/or hyperactivity disorder (AD/HD), internalizing disorders (anxiety and depression), down syndrome and paralyzed individuals [ 15 , 17 – 25 ]. Utilizing some of the most common and considered ML models are support vector machine (SVM), decision trees, random forest (RF), and neural network (NN), these studies have also looked in to comparing the variances in the classification performance of ML models, training recalibrating dataset combination, and classifying target behavior, movement or psychological and biological markers to multiclass behavior outcomes.…”

“…ML-based behavior studies have sourced and analyzed behavior data from phenotypic, electrodermal activity (EDA), or virtual reality (VR) systems, wearable sensors, standardized scales and instruments or task or performance-based measures [ 15 , 17 – 19 , 21 , 23 – 25 ]. Mobile platforms and apps have also been validated using ML classifiers as a screening method, utilized to embed ML-based scoring models, and to deliver or collect the behavioral assessment, screening tools or data for ML-based analyses [ 9 , 29 – 32 ].…”