Augmentation-Enhanced Deep Learning for Face Detection and Emotion Recognition in Elderly Care Robots

Abstract

Facial recognition and emotion detection are crucial for meaningful interactions between elderly individuals and healthcare robots. These technologies allow the robot to identify and understand facial expressions, adapting its responses accordingly. In our research, we propose an innovative approach that combines CNN-based face detection with LBC (Local Binary Convolution) for deep-learning emotion classification. We use a BlazeFace-based face detection model and a robust LBC-Xception emotion classification model. Additionally, we introduce a Multi-Augmentation data method, including seven basic data augmentation techniques and a combination of image super-resolution (ISR) and Conditional Generating Adversarial Network (cGAN) to enhance system robust-ness and accuracy. This combined approach not only improves accuracy through advanced feature extraction capabilities but also provides a lightweight network , suitable for hardware-constrained devices. To validate our method, we will compare face recognition and classification accuracy using the BlazeFace model against other popular classification models like MobileNetV2, VGG16, and Incep-tionV3. We’ll also compare datasets, including FER2013, FER2013Plus, and our augmented dataset (FER2013-MultiX). Finally, we develop an elderly care robot (ECR) and a robot control web application (RCWA) on the Robot Operating System 2 (ROS2) to evaluate performance under real conditions.