Dance Experience System Using Multiple Kinects

Abstract

Abstract-We present a new method for comparing users' motions captured in real time by multiple Kinect sensors with an expert's movements stored in a DB so as to help users experience and learn how to dance. Recently, lots of experience games where users copy some motions to score have been developed. However, it is difficult to collect users' joint data or to clearly compare movements with one sensor because of blocked body parts and unsuccessful tracking. Therefore, such games cannot be applicable to learning beyond simple in-game experiences. As an alternative, this study proposes a method for using multiple Kinect sensors to combine skeletal data and thus to retrieve motions. Also, we propose a method for directly comparing postures between characters of different body sizes. The direct comparison of postures ensures accuracy as no motion is adapted in the process. The proposed dance experience learning system is suitable for learning how to dance as it is easy to implement, features real-time posture comparison and displays the results of important body parts compared.

Motion Capture of the Human Body Using Multiple Depth Sensors

“…However, they focused on tracking the positions of multiple users instead of retrieving their motions. Similarly, Baek and Kim [22] assigned a center Kinect sensor based on movements of root joints. The system tracks the user's face to determine the center sensor.…”

“…Joint inputs from the other sensors are used to retrieve the skeleton joints that the center one fails to track. Similarly, Baek and Kim [22] assigned a center Kinect sensor based on movements of root joints. They retrieved the postures by mixing the five tracked joint segments.…”

“…In addition, the persistence of the Kinect software (Kinect SDK v2) in continually tracking of hidden joint positions in incorrect positions accumulates tracking errors. However, significant noise can occur when joint values are simply added to determine the mean positions [22]. In our system, a skeleton posture is reconstructed at each frame by estimating the optimal joint positions from the noisy data retrieved from the multiple Kinect sensors.…”

“…10, as the user is instructed to follow expert motion, the training system compares the joint movements to the expert's ones and identifies the similarity level as a percentage at the end of each training session. Throughout the sessions, the user's movements are compared based on the five feature vectors, two for each limb and one for torso, extracted from the skeleton posture [22]. For the motion-based contents, a set of four Kinect sensors, one for each side, are installed and synchronized to capture user motion.…”

Dance motion capture and composition using multiple RGB and depth sensors

“…For dance motion, Kitsikidis et al 6 adopted a hidden conditional random fields (HCRF) classifier to recognize motion patterns fused from multi-Kinects. Baek and Kim 7 presented a similar approach for combining the postures, which included mixing the five joint segments tracked by the multi-Kinects system. However, the dance movements in these approaches are slow and simple, while our system mainly targets more dynamic motions such as ballet, modern, and K-pop dances.…”