Identity-Preserving Face Recovery from Stylized Portraits

Shiri, Fatemeh; Yu, Xin; Porikli, Fatih; Hartley, Richard; Koniusz, Piotr

doi:10.1007/s11263-019-01169-1

Cited by 22 publications

(9 citation statements)

References 71 publications

(202 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6(d)). 3) Feature-wise identity similarity loss: Identity preservation is one of the most important goals in face hallucination [57]. Thus, we adopt the identity similarity loss L id to calculate the Euclidean distance between the high-level features of the hallucinated face and the ground truth one, enabling the identity preserving ability of VividGAN.…”

Section: Loss Termsmentioning

confidence: 99%

Face Hallucination With Finishing Touches

Zhang

Tsang

et al. 2021

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

Obtaining a high-quality frontal face image from a low-resolution (LR) non-frontal face image is primarily important for many facial analysis applications. However, mainstreams either focus on super-resolving near-frontal LR faces or frontalizing non-frontal high-resolution (HR) faces. It is desirable to perform both tasks seamlessly for daily-life unconstrained face images. In this paper, we present a novel Vivid Face Hallucination Generative Adversarial Network (VividGAN) devised for simultaneously super-resolving and frontalizing tiny non-frontal face images. VividGAN consists of a Vivid Face Hallucination Network (Vivid-FHnet) and two discriminators, i.e., Coarse-D and Fine-D. The Vivid-FHnet first generates a coarse frontal HR face and then makes use of the structure prior, i.e., fine-grained facial components, to achieve a fine frontal HR face image. Specifically, we propose a facial component-aware module, which adopts the facial geometry guidance as clues to accurately align and merge the coarse frontal HR face and prior information. Meanwhile, the two-level discriminators are designed to capture both the global outline of the face as well as detailed facial characteristics. The Coarse-D enforces the coarse hallucinated faces to be upright and complete; while the Fine-D focuses on the fine hallucinated ones for sharper details. Extensive experiments demonstrate that our VividGAN achieves photo-realistic frontal HR faces, reaching superior performance in downstream tasks, i.e., face recognition and expression classification, compared with other state-of-theart methods.

show abstract

Section: Loss Termsmentioning

confidence: 99%

Face Hallucination With Finishing Touches

Zhang

Tsang

et al. 2021

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…may use Bag-of-Words on hand-crafted descriptors for an alignment task [49,50], or form positive and negative sampling for a contrastive learning strategy [61,62,63]. GAN-based pipelines [18,42,43] also perform self-supervision by generator-discriminator competition.…”

Section: Related Workmentioning

confidence: 99%

Simple Dialogue System with AUDITED

Tas¹,

Koniusz²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We devise a multimodal conversation system for dialogue utterances composed of text, image or both modalities. We leverage Auxiliary UnsuperviseD vIsual and TExtual Data (AUDITED). To improve the performance of text-based task, we utilize translations of target sentences from English to French to form the assisted supervision. For the image-based task, we employ the DeepFashion dataset in which we seek nearest neighbor images of positive and negative target images of the MMD data. These nearest neighbors form the nearest neighbor embedding providing an external context for target images. We form two methods to create neighbor embedding vectors, namely Neighbor Embedding by Hard Assignment (NEHA) and Neighbor Embedding by Soft Assignment (NESA) which generate context subspaces per target image. Subsequently, these subspaces are learnt by our pipeline as a context for the target data. We also propose a discriminator which switches between the image-and text-based tasks. We show improvements over baselines on the large-scale Multimodal Dialogue Dataset (MMD) and SIMMC.

show abstract

“…In the seminal work (Goodfellow et al, 2014), the generative adversarial network (GAN) is introduced to generate realistic-looking images from random noise inputs. GANs have achieved impressive results in image generation (Denton et al, 2015), image editing (Zhu et al, 2016;Zhang et al, 2019a), representation learning (Mathieu et al, 2016), image super-resolution (Ledig et al, 2017;Song et al, 2019) and style transfer (Isola et al, 2017;Shiri et al, 2019). Recently, GANs have been successfully applied to super-resolution (SRGAN) (Ledig et al, 2017), leading to impressive and promising results.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

Multi-task Generative Adversarial Network for Detecting Small Objects in the Wild

et al. 2020

View full text Add to dashboard Cite

Object detection results have been rapidly improved over a short period of time with the development of deep convolutional neural networks (DC-NNs). Although impressive results have been achieved on large/medium sized objects, the performance on small objects is far from satisfactory and one of remaining open challenges is detecting small object in unconstrained conditions (e.g. COCO and WIDER FACE benchmarks). The reason is that small objects usually lack sufficient detailed appearance information, which can distinguish them from the backgrounds or similar objects. To deal with the small object detection problem, in this paper, we propose an end-to-end multi-task generative adversarial network (MTGAN), which is a general framework. In the MTGAN, the generator is a super-resolution network, which can up-sample small blurred images into fine-scale ones and recover detailed information for more accurate detection. The discriminator is a multitask network, which describes each inputted image patch with a real/fake score, object category scores, and bound- † Equal contribution.

show abstract

Identity-Preserving Face Recovery from Stylized Portraits

Cited by 22 publications

References 71 publications

Face Hallucination With Finishing Touches

Face Hallucination With Finishing Touches

Simple Dialogue System with AUDITED

Multi-task Generative Adversarial Network for Detecting Small Objects in the Wild

Contact Info

Product

Resources

About