PoiseNet: Dealing With Data Imbalance in DensePose

GC RCNN: Learning Interdependent IUV for Geodesic Consistent DensePose

“…GC RCNN outperforms the DensePose RCNN by 3.3% and 1.5% with ResNet-50 and ResNet-101-FPN-DL as the backbone, respectively. Compared with PoiseNet [15], which employs a balanced training strategy, GC RCNN achieves 64.7%, suppressing PoiseNet by 1.9% AP gains with our interdependent IUV training strategy. QANet [27] combined the bound box score, mIoU score, and pixel score to rank the proposal and achieved 63.7% AP, while GC RCNN achieved 64.7% AP with the original scoring criterion.…”

“…However, it results in the predicted UV fitting the wrong regression function once a surface is misclassified. Several outstanding works solve this problem by introducing enhancements to improve surface classification, such as multi-scale feature fusion [10], [11], knowledge transfer [12], and re-balanced strategies [15]. Multi-scale feature fusion is an effective way to obtain instance details.…”

“…Data imbalance is one of the major factors limiting the algorithm performance of DensePose. PoiseNet [15] proposed the adaptive equalization loss and block-based landmark localization to re-balance inter-class and intra-class learning. Despite the advantages offered by current technology, achieving optimal surface classification remains a formidable challenge.…”

“…DensePose effectively bridges the dimensional divide by establishing the correspondences between two-dimensional image pixels and three-dimensional vertices on the SMPL human mesh [9]. Through implementing various enhancement modules, such as multi-scale feature fusion [10], [11], knowledge transfer [12], data augmentation [13], [14], re-balanced strategies [15], and quality estimation [16], DensePose has witnessed significant strides. Nonetheless, constrained by the discretized IUV optimization strategy, DensePose still suffers from inconsistency between training strategies and task objectives.…”