No abstract

In the context in which it was demonstrated that humanoid robots are efficient in helping children diagnosed with autism in exploring their affective state, this paper underlines and proves the efficiency of a previously developed machine learning-based mobile application called PandaSays, which was improved and integrated with an Alpha 1 Pro robot, and discusses performance evaluations using deep convolutional neural networks and residual neural networks. The model trained with MobileNet convolutional neural network had an accuracy of 56.25%, performing better than ResNet50 and VGG16. A strategy for commanding the Alpha 1 Pro robot without its native application was also established and a robot module was developed that includes the communication protocols with the application PandaSays. The output of the machine learning algorithm involved in PandaSays is sent to the humanoid robot to execute some actions as singing, dancing, and so on. Alpha 1 Pro has its own programming language—Blockly—and, in order to give the robot specific commands, Bluetooth programming is used, with the help of a Raspberry Pi. Therefore, the robot motions can be controlled based on the corresponding protocols. The tests have proved the robustness of the whole solution.

This paper continues the research of the previous work, regarding PandaSays mobile application, having its main purpose to detect the affective state of the child from his drawings, using MobileNet neural network. Children diagnosed with autism spectrum disorder, have difficulties in expressing their feelings and communicating with others. The purpose of PandaSays mobile application, is to help parents and tutors that have children diagnosed with autism, to communicate better with them and to understand their feelings. The main goal was to improve the model’s accuracy, trained with MobileNet neural network, which reached the value of 84.583%. For training the model, it was used Python programming language. The study focuses further on accessibility and its importance to children diagnosed with autism. Relevant screenshots of the mobile application are presented, in order to indicate that the application follows the accessibility guidelines and rules. Finally, there is presented the interaction with Marty robot and the efficiency of mobile application’s drawing prediction.