Conservative Wasserstein Training for Pose Estimation

Abstract

This paper targets the task with discrete and periodic class labels (e.g., pose/orientation estimation) in the context of deep learning. The commonly used cross-entropy or regression loss is not well matched to this problem as they ignore the periodic nature of the labels and the class similarity, or assume labels are continuous value. We propose to incorporate inter-class correlations in a Wasserstein training framework by pre-defining (i.e., using arc length of a circle) or adaptively learning the ground metric. We extend the ground metric as a linear, convex or concave increasing function w.r.t. arc length from an optimization perspective. We also propose to construct the conservative target labels which model the inlier and outlier noises using a wrapped unimodal-uniform mixture distribution. Unlike the one-hot setting, the conservative label makes the computation of Wasserstein distance more challenging. We systematically conclude the practical closed-form solution of Wasserstein distance for pose data with either one-hot or conservative target label. We evaluate our method on head, body, vehicle and 3D object pose benchmarks with exhaustive ablation studies. The Wasserstein loss obtaining superior performance over the current methods, especially using convex mapping function for ground metric, conservative label, and closed-form solution.

Embedding Semantic Hierarchy in Discrete Optimal Transport for Risk Minimization

“…In this paper, we resort to the optimal transport distance as an alternative for empirical risk minimization [9,10,11,12]. With the low-cost modification of the loss function perspective, our solution can be added on any up-to-date general deep networks in a plug-and-play fashion.…”

Identity-aware Facial Expression Recognition in Compressed Video

“…However, the previous works only consider the image domain, and the spatiotemporal FER does not significantly outperforms aggregation methods [2]. To the best of our knowledge, this is the first effort to investigate the compressed video FER, which is orthogonal to these advantages and can be easily added to each other [37], [38], [39], [40], [41]. Video compression Usually, the video codecs separates a video into several Group Of Pictures (GOP).…”

Deep 3D-CNN for Depression Diagnosis with Facial Video Recording of Self-Rating Depression Scale Questionnaire

“…With the fast development of deep learning for recognition [11], [12], [13], [14], [15], [16], [17], [18], a hierarchical approach is developed for automatically interpreting depression based on the SDS assessment, its associated FE, and action video recording, among other things. To be more specific, we effectively extract the temporal information from each question-wise video by adjusting the 3D convolutional neural networks to the particular question (3D-CNN) [19].…”