Facelet-Bank for Fast Portrait Manipulation

Chen, Ying-Cong; Lin, Huaijia; Shu, Michelle; Li, Ruiyu; Xin, Tao; Shen, Xiaoyong; Ye, Yangang; Jia, Jiaya

doi:10.1109/cvpr.2018.00373

Cited by 47 publications

(28 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…All images were resized to 512 × 512. To obtain the ground‐truth image for each face image, we used facial hair, older, and younger images obtained through the facelet‐bank algorithm. Because the trained facelet‐bank algorithm can be also regarded as an image processing operator, we evaluated how well the proposed network (SBL) and conventional networks (DBL, DGF) can estimate each image processing operator.…”

Section: Experiments Resultsmentioning

confidence: 99%

“…However, the CNN can linearize the nonlinear manifold into Euclidean subspace through the deep feature . Therefore, when extracting a rich representation (deep feature) in the encoding, the image processing operator Ο(·) can be replaced by the following simple linear shift:

normalΦ ((), y) = normalΦ ((), x) + Δν,

where Φ (·) is the deep space, and Δν is the image processing operator in deep space.…”

Section: Proposed Multitask Bilateral Learning Networkmentioning

confidence: 99%

See 1 more Smart Citation

Multitask bilateral learning for real‐time image enhancement

et al. 2019

View full text Add to dashboard Cite

Nowadays, deep neural networks (DNNs) for image processing are becoming more complex; thus, reducing computational cost is increasingly important. This study highlights the construction of a DNN for real‐time image processing, training various image processing operators efficiently through multitask learning. For real‐time image processing, the proposed algorithm takes a joint upsampling approach through bilateral guided upsampling. For multitask learning, the overall network is based on an encoder‐decoder architecture, which consists of encoding, processing, and decoding components, in which the encoding and decoding components are shared by all the image processing operators. In the processing component, a semantic guidance map, which contains processing information for each image processing operator, is estimated using simple linear shifts of the shared deep features. Through these components, the proposed algorithm requires an increase of only 5% in the number of parameters to add another image processing operator and achieves faster and higher performance than that of deep‐learning‐based joint upsampling methods in local image processing as well as global image processing.

show abstract

Section: Experiments Resultsmentioning

confidence: 99%

normalΦ ((), y) = normalΦ ((), x) + Δν,

where Φ (·) is the deep space, and Δν is the image processing operator in deep space.…”

Section: Proposed Multitask Bilateral Learning Networkmentioning

confidence: 99%

Multitask bilateral learning for real‐time image enhancement

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Understanding the relationship between the changes in the image space and the consequences in the deep feature space is still a challenge. Some research effort has been made to manipulate the deep features to tackle the variation in the image space [43], [44], [45], [46], [47], recently. Wen et al [43] introduced a latent factor fully connected (LF-FC) layer (a linear transformation matrix) to extract the age-invariant deep features from convolutional features for aging face recognition.…”

Section: Deep Feature Manipulationmentioning

confidence: 99%

“…Chen et al [45] fed the intermediate deep feature into multiple convolution filter banks to perform image style transfer. Chen et al [46] employ multiple fully convolutional layers to manipulate the middle-level convolutional representations for face portrait transfer.…”

Section: Deep Feature Manipulationmentioning

confidence: 99%

Boosting Occluded Image Classification via Subspace Decomposition-Based Estimation of Deep Features

Cen

Wang

2020

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Classification of partially occluded images is a highly challenging computer vision problem even for the cutting edge deep learning technologies. To achieve a robust image classification for occluded images, this paper proposes a novel scheme using subspace decomposition based estimation (SDBE). The proposed SDBE-based classification scheme first employs a base convolutional neural network to extract the deep feature vector (DFV) and then utilizes the SDBE to compute the DFV of the original occlusion-free image for classification. The SDBE is performed by projecting the DFV of the occluded image onto the linear span of a class dictionary (CD) along the linear span of an occlusion error dictionary (OED). The CD and OED are constructed respectively by concatenating the DFVs of a training set and the occlusion error vectors of an extra set of image pairs. Two implementations of the SDBE are studied in this paper: the l 1 -norm and the squared l 2 -norm regularized least-squares estimates. By employing the ResNet-152, pre-trained on the ILSVRC2012 training set, as the base network, the proposed SBDE-based classification scheme is extensively evaluated on the Caltech-101 and ILSVRC2012 datasets. Extensive experimental results demonstrate that the proposed SDBE-based scheme dramatically boosts the classification accuracy for occluded images, and achieves around 22.25% increase in classification accuracy under 20% occlusion on the ILSVRC2012 dataset.

show abstract

“…hair color, expression, gender and age), and add virtual makeup to human faces. In recent years, face editing has attracted great interests in computer vision fields [1,22,16]. Several image-to-image translation methods [8,27,24] have achieved facial attributes and expressions manipulation on single or multiple domains.…”

Section: Introductionmentioning

confidence: 99%

Texture Deformation Based Generative Adversarial Networks for Multi-domain Face Editing

Chen

Xie

Jia

et al. 2019

PRICAI 2019: Trends in Artificial Intelligence

View full text Add to dashboard Cite

Despite the significant success in image-to-image translation and latent representation based facial attribute editing and expression synthesis, the existing approaches still have limitations in the sharpness of details, distinct image translation and identity preservation. To address these issues, we propose a Texture Deformation Based GAN, namely TDB-GAN, to disentangle texture from original image and transfers domains based on the extracted texture. The approach utilizes the texture to transfer facial attributes and expressions without the consideration of the object pose. This leads to shaper details and more distinct visual effect of the synthesized faces. In addition, it brings the faster convergence during training. The effectiveness of the proposed method is validated through extensive ablation studies. We also evaluate our approach qualitatively and quantitatively on facial attribute and facial expression synthesis. The results on both the CelebA and RaFD datasets suggest that Texture Deformation Based GAN achieves better performance.

show abstract

Facelet-Bank for Fast Portrait Manipulation

Cited by 47 publications

References 24 publications

Multitask bilateral learning for real‐time image enhancement

Multitask bilateral learning for real‐time image enhancement

Boosting Occluded Image Classification via Subspace Decomposition-Based Estimation of Deep Features

Texture Deformation Based Generative Adversarial Networks for Multi-domain Face Editing

Contact Info

Product

Resources

About