Background Counting the repetition of human exercise and physical rehabilitation is a common task in rehabilitation and exercise training. The existing vision-based repetition counting methods less emphasize the concurrent motions in the same video. Methods This work analyzed the spectrogram of the pose estimation result to count the repetition. Besides from the public datasets. This work also collected exercise videos from 11 adults to verify the proposed method is capable for handling concurrent motion and different view angles. Results The presented method was validated on the University of Idaho Physical Rehabilitation Movements Data Set (UI-PRMD) and MM-fit dataset. The overall mean absolute error (MAE) for MM-fit was 0.06 with off-by-one Accuracy (OBOA) 0.94. As for UI-PRMD dataset, MAE was 0.06 with OBOA 0.95. We have also tested the performance in a variety of camera locations and concurrent motions with 57 skeleton time-series video with overall MAE 0.07 and OBOA 0.91. Conclusion The proposed method provides a view-angle and motion agnostic concurrent motion counting. This method can potentially use in large-scale remote rehabilitation and exercise training with only one camera.