Detection of 3D Human Posture Based on Improved Mediapipe

Lin, Yiqiao; Jiao, Xiaohui; Zhao, Lei

doi:10.4236/jcc.2023.112008

Cited by 20 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…With most modern cameras, 60 FPS is standard. Successful motion tracking has been presented for full body posture detection [16] and depth inclusive hand motor skill assessment [20] with 30 FPS. Higher FPS can also provide more data for key-point and pose extraction, leading to increased number of data points and thus, accuracy.…”

Section: Discussionmentioning

confidence: 99%

“…This pipeline has been used primarily for human pose estimation [7], object detection [8], image segmentation [9], with applications such as sign language recognition [10], land-mark detection for augmented reality solutions [11], interactive virtual reality solutions [12], and visually enabled robotic arm control [13] on embedded devices, websites and applications. Recent studies have focused on the use of MediaPipe in clinical research settings, with applications in rehabilitation [14], physiotherapy [15], posture recognition [16], human gait analysis [17], as well as diagnostics in telehealth [18]. Outcome parameters of research studies in these domains include trajectory analysis, range of motion analysis, task based assessment, and accuracy of classification models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Quantifying similarities between MediaPipe and a known standard for tracking 2D hand trajectories

Wagh,

Scott,

Kraeutner

2023

Preprint

View full text Add to dashboard Cite

Marker-less motion tracking methods have promise for use in a range of domains, including clinical settings where traditional marker-based systems for human pose estimation is not feasible. MediaPipe is an artificial intelligence-based system that offers a markerless, lightweight approach to motion capture, and encompasses MediaPipe Hands, for recognition of hand landmarks. However, the accuracy of MediaPipe for tracking fine upper limb movements involving the hand has not been explored. Here we aimed to evaluate 2-dimensional accuracy of MediaPipe against a known standard. Participants (N = 10) performed trials in blocks of a touchscreen-based shape-tracing task. Each trial was simultaneously captured by a video camera. Trajectories for each trial were extracted from the touchscreen and compared to those predicted by MediaPipe. Specifically, following re-sampling, normalization, and Procrustes transformations, root mean squared error (RMSE; primary outcome measure) was calculated for coordinates generated by MediaPipe vs. the touchscreen computer. Resultant mean RMSE was 0.28 +/-0.064 normalized px. Equivalence testing revealed that accuracy differed between MediaPipe and the touchscreen, but that the true difference was between 0-0.30 normalized px (t(114) = -3.02,p= 0.002). Overall, we quantify similarities between MediaPipe and a known standard for tracking fine upper limb movements, informing applications of MediaPipe in a domains such as clinical and research settings. Future work should address accuracy in 3-dimensions to further validate the use of MediaPipe in such domains.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Quantifying similarities between MediaPipe and a known standard for tracking 2D hand trajectories

Wagh,

Scott,

Kraeutner

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…It is also commonly used by professional ML practitioners. MediaPipe provides numerous detection functions including full-body detection, palm tracking, facial tracking, posture detection, face mesh detection, and object detection [18]; hence, it has been widely applied in various fields [19][20][21][22][23][24]. The present study used MediaPipe Pose to detect the posture of swimmers.…”

Section: Application Of ML For Human Skeleton Detectionmentioning

confidence: 99%

Using Machine-Learning Technology to Implement a Nonhardware and Inexpensive Posture Detection System for Analyzing Body Posture Angles in Front Crawl Swimming

Yu-Hung Hsu,

Yu-Hung Hsu

2024

Journal of Computers

View full text Add to dashboard Cite

<p>Swimming is a sport that relies heavily on motor skills. Inability to maintain adequate bodily balance in water prevents swimmers from remaining afloat and propelling themselves. Due to the difficulty of attaching reflective stickers or LED (light-emitting diode) emitters to the body while adjusting the swimming posture, it is not possible to capture the posture like during a bicycle fitting; the refraction of water also affects the detection of the body’s posture. Addressing the shortcomings, this study developed a low-cost nonhardware posture detection system based on machine-learning models in MediaPipe. The system provides real-time and post analyses of posture angles and posture lines during front crawl swimming, thereby facilitating observation of the relationship between angle at which the arm enters the water and the body horizon. Two participants practicing front crawl were invited to test the proposed system. The experimental results confirmed that the proposed system provides effective detection and analyses of posture lines and angles in swimmers. The study also proposed the algorithm for optimizing posture angle detection to solve the problems of posture line distortion and angle calculation errors that arise when MediaPipe was used to detect a human skeleton above a water line. The system does not require the installation of hardware and is inexpensive to deploy, and it can be widely applied in front crawl swimming lessons to help learners adjust their arm’s entry angle and body horizon to reduce forward drag and increase speed.</p> <p> </p>

show abstract

“…Detector is used to find that ROI point there 33 points are used to segregate the human structure and Tracker follows these points and link them up to create a skeletal structure that provides the movement form of human by using a ML model to train on the already existing movements to see the similarities and compare. MediaPipe is lightweight and optimized for on-device execution, so it can be processed at the point of real-time use [9]. Detecting key points Euclidean distance between two key points (x1, y1) and (x2, y2).…”

Section: Media Pipelinementioning

confidence: 99%

Drones in Human Aid and Disaster Relief using Thermal Imaging Technology

Thiyagarajan S,

Arunmozhiselvam B,

Akash G

et al. 2024

Int Res J Adv Engg Mgt

View full text Add to dashboard Cite

The rapidly evolving landscape of medical imaging necessitates innovative approaches to enhance diagnostic accuracy, particularly in liver and tumor segmentation. Current methodologies often grapple with limitations such as the need for extensive labeled datasets and the challenge of generalizing across diverse pathological presentations. Addressing these constraints, this study introduces a novel deep learning model that amalgamates the prowess of U-Net architecture, Generative Adversarial Networks (GANs), and Monte Carlo Dropout techniques, each contributing uniquely to overcome the existing barriers. Predominantly, the traditional segmentation models are hindered by their dependency on large, annotated datasets, which are not only scarce but also labor-intensive to produce. Further, these models frequently struggle to maintain consistent performance across varying imaging conditions, a pivotal requirement in medical diagnostics. To surmount these challenges, our proposed model employs a fine-tuning methodology using a U-Net architecture pre-trained on extensive medical image datasets like NIH Chest X-ray or MIMIC-CXR. This strategy leverages the pre-existing knowledge within the network, significantly enhancing the model's ability to discern and adapt to the specific characteristics inherent in liver and tumor images & samples. In a parallel vein, the model harnesses the capabilities of GANs for data augmentation, generating synthetic yet realistic medical images. This innovative use of GANs addresses the issue of dataset variability, equipping the model to generalize more effectively across a spectrum of liver and tumor appearances, ultimately bolstering its robustness. This approach is particularly advantageous in handling cases with diverse lighting, angles, or pathological conditions, traditionally a stumbling block for segmentation models. Moreover, the incorporation of Monte Carlo Dropout for uncertainty estimation marks a significant stride in the realm of clinical applicability. This technique yields pixel-wise uncertainty maps, offering invaluable insights into the model's confidence levels in its predictions. Such transparency is crucial in clinical settings, where understanding the model's certainty can guide more informed and cautious decision-making, particularly in scenarios where the cost of erroneous segmentation is high for different scenarios. Empirical evaluation of this composite model on Kaggle datasets has demonstrated its superior performance, evidenced by improvements in precision (3.9%), accuracy (4.9%), recall (4.5%), AUC (8.5%), and a reduction in delay (3.5%) compared to existing methods. The culmination of these advancements not only enhances the efficacy of liver and tumor segmentation but also paves the way for broader applications in medical imaging diagnostics.

show abstract

Detection of 3D Human Posture Based on Improved Mediapipe

Cited by 20 publications

References 7 publications

Quantifying similarities between MediaPipe and a known standard for tracking 2D hand trajectories

Quantifying similarities between MediaPipe and a known standard for tracking 2D hand trajectories

Using Machine-Learning Technology to Implement a Nonhardware and Inexpensive Posture Detection System for Analyzing Body Posture Angles in Front Crawl Swimming

Drones in Human Aid and Disaster Relief using Thermal Imaging Technology

Contact Info

Product

Resources

About