An Open-Source Platform for Human Pose Estimation and Tracking Using a Heterogeneous Multi-Sensor System

Patil, Ashok Kumar; Balasubramanyam, Adithya; Ryu, Jaeyeong; Chakravarthi, Bharatesh; Chai, Young Ho

doi:10.3390/s21072340

Cited by 19 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At present, researchers are attempting to design complex sensing environments that use multimodality, because a complex system can collect more accurate motion data [19]. We gather human motion data using a multimodal sensing system [20]. The system was composed of one Lidar sensor and ten IMU sensors.…”

Section: Related Work a Sensing System And Benchmark Data Setmentioning

confidence: 99%

Angular Features-Based Human Action Recognition System for a Real Application With Subtle Unit Actions

et al. 2022

Self Cite

View full text Add to dashboard Cite

Human action recognition (HAR) technology is receiving considerable attention in the field of human-computer interaction. We present a HAR system that works stably in real-world applications. In realworld applications, the HAR system needs to identify detailed actions for specific purposes, and the action data includes many variations. Accordingly, we conducted three experiments. First, we tested our recognition system's performance on the UTD-MHAD dataset. We compared our system's accuracy with results from previous research and confirmed that our system achieves a 91% average performance among recognition systems. Furthermore, we hypothesized the use of a HAR system to detect burglary. In the second experiment, we compared the existing benchmark data with our crime detection dataset. We recognized the test scenarios' data by using the recognition system trained by each dataset. The recognition system trained by our dataset achieved higher accuracy than the past benchmark dataset. The results show that the training data should contain detailed actions for a real application. In the third experiment, we tried to find the motion data type that stably recognizes action regardless of data variation. In a real application, the action data are changed by people. Thus, we introduced variations in the action data using the cross-subject protocol and moving area setting. We trained the recognition system using each position and angle data. In addition, we compared the accuracy of each system. We found that the angle format results in better accuracy because the angle data are beneficial for converting the action variation into a consistent pattern.

show abstract

Section: Related Work a Sensing System And Benchmark Data Setmentioning

confidence: 99%

Angular Features-Based Human Action Recognition System for a Real Application With Subtle Unit Actions

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…The user performing the boxing action was well within the LiDAR's field of view (1.5 m away from LiDAR system and within a 5 m sensing range), as shown in Figure 11. Furthermore, we estimate the body segment's orientation and joint position using an open-source framework [33]. In this comparative study, we used the estimated orientations and positions from LiDAR measurements and IMU sensors as the ground truth.…”

Section: A Experimental Setupmentioning

confidence: 99%

“…A vector-based joint position estimation framework [33] was used to evaluate the proposed system's position estimation accuracy. The fullbody skeleton joint positions tracked from the LiDAR's point cloud were fused with those estimated using the IMU sensors.…”

Section: ) Position Accuracy Comparisonmentioning

confidence: 99%

Scenario-Based Sensed Human Motion Editing and Validation Through the Motion-Sphere

et al. 2022

Self Cite

View full text Add to dashboard Cite

Synthesizing realistic human motion data using a real-time motion capture system in a controlled environment is a critical challenge. In addition, effectively manipulating the existing motion data is another primary concern and using such modified data in human motion analysis and activity recognition systems are prone to errors. This paper presents a simplistic and comprehensive system to effortlessly author, edit, and validate human motion data. The system enables a naive user to edit the existing motion data interactively using a humanoid model in a three-dimensional space, based on user-defined scenarios and synthesize numerous variations of the motion sequences. A modular concept of scenario-based sensed unit motion editing has been adopted to demonstrate the proposed system. We employed an efficient analytical kinematic and constraint solver to enforce the inherent body joint limits and external constraints while editing to synthesize complete and meaningful motion sequences. Furthermore, we substantiated the proposed sensed unit motion editing framework through a visual validation study using an open-source intuitive visualization tool called the Motion-Sphere. Finally, we compared the resultant synthesized motion against the real-time motion capture system data to verify the body segments' orientation and position accuracy deviations.

show abstract

“…To date, automatic methods that reconstruct a human body in three dimensions (3D) have been explored to capture human pose movements. Typically, research on classifying human body postures involves estimating a human's 3D pose and shape from one or more color images [19][20][21][22][23][24][25][26]. The pose estimation methods demonstrated to date have shown impressive results in estimating a 3D human that fits well with the image features extracted from camera views.…”

Section: Introductionmentioning

confidence: 99%

Lightweight Three-Dimensional Pose and Joint Center Estimation Model for Rehabilitation Therapy

Kim,

Ku,

Yang

et al. 2023

Electronics

View full text Add to dashboard Cite

In this study, we proposed a novel transformer-based model with independent tokens for estimating three-dimensional (3D) human pose and shape from monocular videos, specifically focusing on its application in rehabilitation therapy. The main objective is to recover pixel-aligned rehabilitation-customized 3D human poses and body shapes directly from monocular images or videos, which is a challenging task owing to inherent ambiguity. Existing human pose estimation methods heavily rely on the initialized mean pose and shape as prior estimates and employ parameter regression with iterative error feedback. However, video-based approaches face difficulties capturing joint-level rotational motion and ensuring local temporal consistency despite enhancing single-frame features by modeling the overall changes in the image-level features. To address these limitations, we introduce two types of characterization tokens specifically designed for rehabilitation therapy: joint rotation and camera tokens. These tokens progressively interact with the image features through the transformer layers and encode prior knowledge of human 3D joint rotations (i.e., position information derived from large-scale data). By updating these tokens, we can estimate the SMPL parameters for a given image. Furthermore, we incorporate a temporal model that effectively captures the rotational temporal information of each joint, thereby reducing jitters in local parts. The performance of our method is comparable with those of the current best-performing models. In addition, we present the structural differences among the models to create a pose classification model for rehabilitation. We leveraged ResNet-50 and transformer architectures to achieve a remarkable PA-MPJPE of 49.0 mm for the 3DPW dataset.

show abstract

An Open-Source Platform for Human Pose Estimation and Tracking Using a Heterogeneous Multi-Sensor System

Cited by 19 publications

References 35 publications

Angular Features-Based Human Action Recognition System for a Real Application With Subtle Unit Actions

Angular Features-Based Human Action Recognition System for a Real Application With Subtle Unit Actions

Scenario-Based Sensed Human Motion Editing and Validation Through the Motion-Sphere

Lightweight Three-Dimensional Pose and Joint Center Estimation Model for Rehabilitation Therapy

Contact Info

Product

Resources

About