LOTR: Face Landmark Localization Using Localization Transformer

Watchareeruetai, Ukrit; Sommanna, Benjaphan; Jain, Sanjana; Noinongyao, Pavit; Ganguly, Ankush; Samacoïts, Aubin; Earp, Samuel W. F.; Sritrakool, Nakarin

doi:10.1109/access.2022.3149380

Cited by 14 publications

(12 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare DDT with the most competitive methods on WFLW test set. As Table 2 depicts, DDT exceeds most current CBR methods such as LAB [20], Wing [13], SDFL [28], and LOTR [25] by 0.17–1.13 in terms of NME. Besides, DDT is on a par with SLPT [27] which is a SOTA CBR method in terms of NME.…”

Section: Resultsmentioning

confidence: 99%

“…CBR methods [13,14,20,25,28] often add an FC layer (which has S � 2 neurons) to the last layer of the network, in which S denotes the number of landmarks. Compared with HBR methods, CBR methods directly output the coordinates of landmarks; therefore, CBR methods are differentiable globally and do not have the quantisation error.…”

Section: Cbr Methods-based Cnnmentioning

confidence: 99%

“…DLQ versus FLQ. Compared with other methods based on the transformer, such as SLPT [27] and LOTR [25], DDT exploits DLQs which are not pre‐defined class. Class and coordinate are predicted by the regression network together.…”

Section: Ablation Studymentioning

confidence: 99%

“…Besides, face alignment methods are also divided into heatmap‐based regression (HBR) methods [15–19, 22, 23] and coordinate‐based regression (CBR) methods [13, 14, 20, 25, 28]. For HBR methods, the network's output is multiple heatmaps, each of which is responsible for predicting one landmark.…”

Section: Introductionmentioning

confidence: 99%

“…According to the network structure, current face alignment methods can be divided into CNN-based methods [13][14][15][16][17][18][19][20][21][22][23][24] and transformer-based methods [25][26][27]. Although CNN-based methods have a not bad performance, they only focus on features in small patches like 5 � 5 and fail to capture long distance features.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Dynamic deformable transformer for end‐to‐end face alignment

Han

Yang

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

Heatmap‐based regression (HBR) methods have dominated for a long time in the face alignment field while these methods need complex design and post‐processing. In this study, the authors propose an end‐to‐end and simple enough coordinate‐based regression (CBR) method called Dynamic Deformable Transformer (DDT) for face alignment. Unlike general pre‐defined landmark queries, DDT uses Dynamic Landmark Queries (DLQs) to query landmarks' classes and coordinates together. Besides, DDT adopts a deformable attention mechanism rather than a regular attention mechanism which has a faster convergence speed and lower computational complexity. Experiment results on three mainstream datasets 300W, WFLW, and COFW demonstrate DDT exceeds the state‐of‐the‐art CBR methods by a large margin and is comparable to the current state‐of‐the‐art HBR methods with much less computational complexity.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Cbr Methods-based Cnnmentioning

confidence: 99%

Section: Ablation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Dynamic deformable transformer for end‐to‐end face alignment

Han

Yang

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

show abstract

Motion Transformer for Unsupervised Image Animation

Tao

Wang

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multi-domain Multi-definition Landmark Localization for Small Datasets

Ferman

Bharaj²

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

3D facial landmark localization has proven to be of particular use for applications, such as face tracking, 3D face modeling, and image-based 3D face reconstruction. In the supervised learning case, such methods usually rely on 3D landmark datasets derived from 3DMM-based registration that often lack spatial definition alignment, as compared with that chosen by hand-labeled human consensus, e.g., how are eyebrow landmarks defined? This creates a gap between landmark datasets generated via high-quality 2D human labels and 3DMMs, and it ultimately limits their effectiveness. To address this issue, we introduce a novel semi-supervised learning approach that learns 3D landmarks by directly lifting (visible) hand-labeled 2D landmarks and ensures better definition alignment, without the need for 3D landmark datasets. To lift 2D landmarks to 3D, we leverage 3D-aware GANs for better multi-view consistency learning and inthe-wild multi-frame videos for robust cross-generalization. Empirical experiments demonstrate that our method not only achieves better definition alignment between 2D-3D landmarks but also outperforms other supervised learning 3D landmark localization methods on both 3DMM labeled and photogrammetric ground truth evaluation datasets. Project

show abstract

LOTR: Face Landmark Localization Using Localization Transformer

Cited by 14 publications

References 63 publications

Dynamic deformable transformer for end‐to‐end face alignment

Dynamic deformable transformer for end‐to‐end face alignment

Motion Transformer for Unsupervised Image Animation

Multi-domain Multi-definition Landmark Localization for Small Datasets

Contact Info

Product

Resources

About