The aim of this study was to compare the physiological and psychological effects in the elderly during horticultural and nonhorticultural activities as leisure activities. A total of 58 participants aged 65 or older (29 men and 29 women; average age, 74.0 ± 4.7) whose cognitive function was within the normal range were included in this study. Participants performed four horticultural and four nonhorticultural activities for 2 min, respectively. The study had a cross-over experimental design. Electroencephalography was performed during all the activities. Subjective evaluation of emotions was performed using the Profile of Mood States immediately after each activity. The collected statistical data were analyzed using Duncan’s test as a post-analysis of variance test to verify the differences in the results of electroencephalography and the Profile of Mood States according to the different activities. In the results of the electroencephalography, the relative beta, gamma, low-beta, and ratio of sensorimotor rhythm to theta indices indicate that the degree of brain activity in the prefrontal lobe was high during activities such as washing leaves, transplanting plants, and reading news. The results of the Profile of Mood States showed that during activities such as arranging flowers, transplanting, and washing leaves, the total mood disorder score was lowered, indicating a positive effect on the mood of the participants. This study shows that activities such as washing leaves, transplanting, and reading news have a positive effect on the cognitive function of elderly people by increasing brain activity.