ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search

Xu, Lumin; Guan, Yuanyuan; Jin, Sheng; Li, Wentao; Qian, Chen; Luo, Ping; Ouyang, Wanli; Wang, Xiaogang

doi:10.1109/cvpr46437.2021.01581

Cited by 43 publications

(19 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PoseNAS [92] adopts a Fusionand-Enhancement manner with scale-adaptive fusion cells for pose decoder, and end-to-end searches the pose encoder and pose decoder simultaneously. ViPNAS [93] proposes a spatial-temporal neural architecture search framework for efficient video pose estimation. Our ZoomNAS is different from these NAS methods for human pose estimation on two aspects.…”

Section: Neural Architecture Searchmentioning

confidence: 99%

ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild

Jin

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

This paper investigates the task of 2D whole-body human pose estimation, which aims to localize dense landmarks on the entire human body including body, feet, face, and hands. We propose a single-network approach, termed ZoomNet, to take into account the hierarchical structure of the full human body and solve the scale variation of different body parts. We further propose a neural architecture search framework, termed ZoomNAS, to promote both the accuracy and efficiency of whole-body pose estimation. ZoomNAS jointly searches the model architecture and the connections between different sub-modules, and automatically allocates computational complexity for searched sub-modules. To train and evaluate ZoomNAS, we introduce the first large-scale 2D human whole-body dataset, namely COCO-WholeBody V1.0, which annotates 133 keypoints for in-the-wild images. Extensive experiments demonstrate the effectiveness of ZoomNAS and the significance of COCO-WholeBody V1.0.

show abstract

Section: Neural Architecture Searchmentioning

confidence: 99%

ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild

Jin

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…This approach heavily relies on the object images to be identified, and cannot achieve good results with small objects. Recently, Lumin Su et al introduced ViPNAS, an effective video pose estimation search in both spatial and temporal levels using ResNet-50 and MobileNet v3 as the backbone [19]. Experiments on the COCO2017 and PoseTrack2018 datasets provided high inference speeds without sacrificing accuracy.…”

Section: State Of the Art Overviewmentioning

confidence: 99%

Content-Based Video Big Data Retrieval with Extensive Features and Deep Learning

Phan

Cao

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

In the era of digital media, the rapidly increasing volume and complexity of multimedia data cause many problems in storing, processing, and querying information in a reasonable time. Feature extraction and processing time play an extremely important role in large-scale video retrieval systems and currently receive much attention from researchers. We, therefore, propose an efficient approach to feature extraction on big video datasets using deep learning techniques. It focuses on the main features, including subtitles, speeches, and objects in video frames, by using a combination of three techniques: optical character recognition (OCR), automatic speech recognition (ASR), and object identification with deep learning techniques. We provide three network models developed from networks of Faster R-CNN ResNet, Faster R-CNN Inception ResNet V2, and Single Shot Detector MobileNet V2. The approach is implemented in Spark, the next-generation parallel and distributed computing environment, which reduces the time and space costs of the feature extraction process. Experimental results show that our proposal achieves an accuracy of 96% and a processing time reduction of 50%. This demonstrates the feasibility of the approach for content-based video retrieval systems in a big data context.

show abstract

“…Designing efficient human pose estimators has been intensively studied for practical usage. For extracting 2D pose from images, state-of-the-art methods [22,26,36,50,53] have achieved real-time inference speed. In terms of multiview 3D pose estimation, Bultman et al .…”

Section: Efficient Human Pose Estimationmentioning

confidence: 99%

Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection

Ye¹,

Zhu²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

While the voxel-based methods have achieved promising results for multi-person 3D pose estimation from multi-cameras, they suffer from heavy computation burdens, especially for large scenes. We present Faster VoxelPose to address the challenge by re-projecting the feature volume to the three two-dimensional coordinate planes and estimating X, Y, Z coordinates from them separately. To that end, we first localize each person by a 3D bounding box by estimating a 2D box and its height based on the volume features projected to the xy-plane and z-axis, respectively. Then for each person, we estimate partial joint coordinates from the three coordinate planes separately which are then fused to obtain the final 3D pose. The method is free from costly 3D-CNNs and improves the speed of VoxelPose by ten times and meanwhile achieves competitive accuracy as the state-of-the-art methods, proving its potential in real-time applications.

show abstract

ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search

Cited by 43 publications

References 37 publications

ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild

ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild

Content-Based Video Big Data Retrieval with Extensive Features and Deep Learning

Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection

Contact Info

Product

Resources

About