Robot-assisted activity (RAA) is a non-pharmacological therapy used to treat behavioral and psychological symptoms of dementia. This study investigated the immediate effects of RAA on psychological and neurophysiological indices. Twenty-eight elderly people were assigned to the cognitive decline group ( n = 11) or control group ( n = 17) based on their Mini-Mental State Examination scores. After 5-min RAA sessions that involved patients interacting with a communication robot, patient emotions and mood states were measured, and resting-state EEG activity and salivary cortisol were assessed before and after RAA. We found that compared with those in the control group, participants in the cognitive decline group did not enjoy RAA using the communication robot. This was corroborated by EEG findings indicating decreased relaxation immediately after RAA exposure. These results suggested that participants with cognitive decline had difficulty understanding the contents of communication with the robot. Our results indicated that elderly people who have cognitive decline and use day-service centers are less likely to experience the immediate benefits of RAA, including positive emotions and mental relaxation. To conduct effective RAA for such populations, it may be useful to select a method that is better understood and enjoyed by participants.