THANet: Transferring Human Pose Estimation to Animal Pose Estimation

Liao, Jincheng; Xu, Jianzhong; Shen, Yunhang; Lin, Shaohui

doi:10.3390/electronics12204210

Cited by 3 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The traditional graph structure builds a spatial graph based on body joints in each frame [10,34]. However, this modeling approach only considers the physical constraints between body joints.…”

Section: Joint Hypergraphs Generationmentioning

confidence: 99%

See 1 more Smart Citation

Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos

Cai,

Cheng,

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

Compared to traditional dance, intangible cultural heritage dance often involves the isotropic extension of choreographic actions, utilizing both upper and lower limbs. This characteristic choreography style makes the remote joints lack interaction, consequently reducing accuracy in existing human motion prediction methods. Therefore, we propose a human motion prediction method based on the multi-scale hypergraph convolutional network of the intangible cultural heritage dance video. Firstly, this method inputs the 3D human posture sequence from intangible cultural heritage dance videos. The hypergraph is designed according to the synergistic relationship of the human joints in the intangible cultural heritage dance video, which is used to represent the spatial correlation of the 3D human posture. Then, a multi-scale hypergraph convolutional network is constructed, utilizing multi-scale transformation operators to segment the human skeleton into different scales. This network adopts a graph structure to represent the 3D human posture at different scales, which is then used by the single-scalar fusion operator to spatial features in the 3D human posture sequence are extracted by fusing the feature information of the hypergraph and the multi-scale graph. Finally, the Temporal Graph Transformer network is introduced to capture the temporal dependence among adjacent frames within the time domain. This facilitates the extraction of temporal features from the 3D human posture sequence, ultimately enabling the prediction of future 3D human posture sequences. Experiments show that we achieve the best performance in both short-term and long-term human motion prediction when compared to Motion-Mixer and Motion-Attention algorithms on Human3.6M and 3DPW datasets. In addition, ablation experiments show that our method can predict more precise 3D human pose sequences, even in the presence of isotropic extensions of upper and lower limbs in intangible cultural heritage dance videos. This approach effectively addresses the issue of missing segments in intangible cultural heritage dance videos.

show abstract

Section: Joint Hypergraphs Generationmentioning

confidence: 99%

“…frame [10,34]. However, this modeling approach only considers the physical constraints between body joints.…”

Section: Joint Hypergraphs Generationmentioning

confidence: 99%

Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos

Cai,

Cheng,

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

Automatic identification of the endangered hawksbill sea turtle behavior using deep learning and cross-species transfer learning

Jeantet,

Zondo,

Delvenne

et al. 2024

Journal of Experimental Biology

View full text Add to dashboard Cite

The accelerometer, an onboard sensor, enables remote monitoring of animal posture and movement, allowing researchers to deduce behaviors. Despite the automated analysis capabilities provided by deep learning, data scarcity remains a challenge in ecology. We explored transfer learning to classify behaviors from acceleration data of critically endangered hawksbill sea turtles (Eretmochelys imbricata). Transfer learning reuses a model trained on one task from a large dataset to solve a related task. We applied this method using a model trained on green turtles (Chelonia mydas) and adapted it to identify hawksbill behaviors like swimming, resting, and feeding. We also compared this to a model trained on human activity data. Results showed an 8% and 4% F1-score improvement with transfer learning from green turtle and human datasets, respectively. Transfer learning allows researchers to adapt existing models to their study species, leveraging deep learning and expanding the use of accelerometers for wildlife monitoring.

show abstract

MPE-HRNetL: A Lightweight High-Resolution Network for Multispecies Animal Pose Estimation

Shen,

Jiang,

Luo

et al. 2024

Sensors

View full text Add to dashboard Cite

Animal pose estimation is crucial for animal health assessment, species protection, and behavior analysis. It is an inevitable and unstoppable trend to apply deep learning to animal pose estimation. In many practical application scenarios, pose estimation models must be deployed on edge devices with limited resource. Therefore, it is essential to strike a balance between model complexity and accuracy. To address this issue, we propose a lightweight network model, i.e., MPE-HRNet.L, by improving Lite-HRNet. The improvements are threefold. Firstly, we improve Spatial Pyramid Pooling-Fast and apply it and the improved version to different branches. Secondly, we construct a feature extraction module based on a mixed pooling module and a dual spatial and channel attention mechanism, and take the feature extraction module as the basic module of MPE-HRNet.L. Thirdly, we introduce a feature enhancement stage to enhance important features. The experimental results on the AP-10K dataset and the Animal Pose dataset verify the effectiveness and efficiency of MPE-HRNet.L.

show abstract

THANet: Transferring Human Pose Estimation to Animal Pose Estimation

Cited by 3 publications

References 38 publications

Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos

Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos

Automatic identification of the endangered hawksbill sea turtle behavior using deep learning and cross-species transfer learning

MPE-HRNetL: A Lightweight High-Resolution Network for Multispecies Animal Pose Estimation

Contact Info

Product

Resources

About