With the aging problem becoming more and more serious, muscle pain has become a common symptom. In order to help patients with rehabilitation training, it is necessary to monitor their activities in time. We propose a real-time monitoring method based on wearable devices. This method uses a wireless body area network for health care. Specifically, in the first step, we developed a wearable device based on ZigBee with low cost and low weight. Secondly, if only classifying the action at the current time, it will not meet the requirements of real-time monitoring. So, we design an end-to-end neural network model called ATCRNN to infer the actions to be made by users at the next time according to the data of the past few times. This model uses CNN and RNN to extract the spatial and temporal features of data and captures the context characteristics through self-attention. Finally, four volunteers wore equipment to participate in the experiment. The activity categories in the experiment are walking, sitting down, running, and climbing stairs. The accuracy of behavior inference reached 97% with ATCRNN. The compared results demonstrate that the performance of the proposed approach is superior to the result of other deep learning networks.