LOTR: Face Landmark Localization Using Localization Transformer

Abstract

This paper presents a novel Transformer-based facial landmark localization network named Localization Transformer (LOTR). The proposed framework is a direct coordinate regression approach leveraging a Transformer network to better utilize the spatial information in the feature map. An LOTR model consists of three main modules: 1) a visual backbone that converts an input image into a feature map, 2) a Transformer module that improves the feature representation from the visual backbone, and 3) a landmark prediction head that directly predicts the landmark coordinates from the Transformer's representation. Given cropped-and-aligned face images, the proposed LOTR can be trained end-to-end without requiring any post-processing steps. This paper also introduces the smooth-Wing loss function, which addresses the gradient discontinuity of the Wing loss, leading to better convergence than standard loss functions such as L1, L2, and Wing loss. Experimental results on the JD landmark dataset provided by the First Grand Challenge of 106-Point Facial Landmark Localization indicate the superiority of LOTR over the existing methods on the leaderboard and two recent heatmap-based approaches.

Motion Transformer for Unsupervised Image Animation

“…ViT [9] proposes a transformer encoder architecture for image classification, which directly splits the image into patches and introduces a learnable classification token to aid in performing the task. Recently, several methods have proposed variant forms of transformers for landmark detection [45,18,50]. Our method is motivated by these recent works, in that we regress the object keypoints as well as affine matrices for image animation.…”