As the number of elderly people living alone increases, the need for remote monitoring system is also increasing. The system automatically checks the safety of the elderly and notifies the state to remote areas in case of anomalies. However, how to protect the privacy of the watched person becomes a problem. We propose that skeleton recognition technology is useful to monitor people with high accuracy while protecting the privacy. It can be used not only to investigate his/her posture and motion, but also to selectively analyze the voice emitted by himself/herself. We developed a system that combines skeleton recognition and selective speech recognition by the audio direction. In this paper, we will explain the improvement of our system and report some experiment results.