Estimating Children Engagement Interacting with Robots in Special Education Using Machine Learning

Abstract

The task of child engagement estimation when interacting with a social robot during a special educational procedure is studied. A multimodal machine learning-based methodology for estimating the engagement of the children with learning difficulties, participating in appropriate designed educational scenarios, is proposed. For this purpose, visual and audio data are gathered during the child-robot interaction and processed towards deciding an engaged state of the child or not. Six single and three ensemble machine learning models are examined for their accuracy in providing confident decisions on in-house developed data. The conducted experiments revealed that, using multimodal data and the AdaBoost Decision Tree ensemble model, the children’s engagement can be estimated with 93.33% accuracy. Moreover, an important outcome of this study is the need for explicitly defining the different engagement meanings for each scenario. The results are very promising and put ahead of the research for closed-loop human centric special education activities using social robots.

The key artificial intelligence technologies in early childhood education: a review

“…For example, data mining technologies were used to extract useful information from the behavioural characteristics data (Al-Diabat 2018), the digital library of coded video (Leroy et al 2006), the occurrence of allergies in children and daily upperair observations (El Afandi 2013) and then decision tree (Leroy et al 2006;Chang 2007), fuzzy data mining models (Al-Diabat 2018), and classification algorithms (Abdullah et al 2016;Zhang et al 2009) are used to detect/predict children's symptoms and design personal education and childcare. Machine learning technologies developed algorithms [e.g., AdaBoost (Papakostas et al 2021), random forest (Ahmadi et al 2018)], support vector machine (Liu et al 2016) to identify children's activities such as attention (Papakostas et al 2021), failure (Rasheed et al 2021), anxiety disorder (Carpenter et al 2016;McGinnis et al 2018), ASD (Crippa et al 2015;Liu et al 2016), suicidal behavior (Su et al 2020). Note that although data mining and machine learning in ECE have a few overlaps of data, there are a considerable number of differences between them.…”

“…Visual and audio data are gathered during the child-robot interaction and processed towards deciding an engaged state of children by AdaBoost decision tree (Papakostas et al 2021). Hagenbuchner et al (2015) develop machine learning models for predicting activity types in preschool-aged children.…”

“…• Overfitting of training data is a common challenge for machine learning in ECE when data is massive amounts of noisy and biased data, which can generally be solved by removing outliers data and longer training time (Rasheed et al 2021;McGinnis et al 2018McGinnis et al , 2019). • The absence of good-quality data in ECE is a current challenge for machine learning since it is hard to collect good-quality data from young children (Papakostas et al 2021;Hagenbuchner et al 2015). The operation of removing outliers, filtering missing values, and removing unwanted features can be used for data preprocessing.…”

A Perspective on the Current Situation of Children’s Education and Teachers’ Moral Cultivation Combined with Decision Tree Algorithm

“…Suppose the knowledge obtained by machine learning is applied to the information system of colleges and universities. In that case, the original system will become smarter and help us make better decisions or find more problems so as to improve the quality of education [4][5][6]. To this end, the classic algorithm of machine learning -the decision tree algorithm is applied to the current children's education platform to discover the potential information and rules of the data, to find the key factors affecting the quality of teaching and learning, so as to improve the children's education and teaching management level and teacher ethics.…”

Behavioral Data Analysis of Robot-Assisted Autism Spectrum Disorder (ASD) Interventions Based on Lattice Computing Techniques

“…The objective of the work was to help the robot to determine the time of effective intervention as well as the type of the most beneficial behavior to be induced in the users. Ad hoc feature extraction has also been considered for the estimation in children engagement [24]. With this method, machine learning techniques are employed to determine from multi-modal data whether the child is engaged or not, based on manual annotations by experts.…”