In an aging population, the gradual decline in cognitive, comprehensive, and mobility abilities poses a significant challenge for the independent living of the elderly. Accurate assessment of such abilities becomes crucial, not only for facilitating care-giving but also for policy-making related to elder care. To address this need, this study presents a novel image recognition-based assessment system designed to accurately evaluate the independent living ability of the elderly. The system construction and subsequent real-world applications constitute the initial focus of this study. Significant efforts have been made to detect key points of the skeletal structure in elderly individuals. The skeleton extraction process has been methodically divided into five distinct steps: detection of key points, matching and connecting joint points, generating coordinates of joint points and their connection maps, measuring correlations between key point pairs, and calculating optimal matching results using a bipartite graph approach. In the latter part of the study, an advanced model integrating an attention mechanism with a Graph Convolutional Neural Network (GCNN) has been developed and implemented for elder behavior recognition. The effectiveness and validity of this approach have been assessed through rigorous experimental validation. This study's findings can potentially revolutionize the quality of life assessment for the elderly and provide valuable insights for relevant policy-making. Further research in this direction is deemed necessary for enhancing the assessment system and expanding its applications.