Faced with the situation that the elderly people at home have dangerous behaviors, the study explores various aspects of motion target detection, real-time target tracking and behavioral pose recognition and classification, using behavioral poses in videos as samples. To tackle the challenges in detecting motion targets, a target detection method based on Gaussian mixture model (GMM) and four frame difference method is proposed; A tracking technique incorporating Kalman filter (KF) is investigated to trail the behavioral changes of the elderly in actual time. A seven-layer convolutional neural network (CNN) is constructed to face the problem of inaccurate behavioral pose recognition. Through relevant experimental analyses, the outcomes show that the increased GMM detection way has a complete profile and the accuracy is significantly improved. The KF target tracking technique can trail the object trajectory in actual time and steadily, with the smallest trailing error value of 0.19. The classification accuracy of the CNN pose recognition model is 95.87%, and the pose classification time is 27 seconds. Its performance is superior to the mean shift algorithm, particle filter algorithm, and Cam Shift algorithm in all aspects. When applied in practice, it can accurately identify whether the elderly’s behavior is abnormal and ensure their daily health.