Style transfer for headshot portraits

Shih, YiChang; Paris, Sylvain; Barnes, Connelly; Freeman, William T.; Durand, Frédo

doi:10.1145/2601097.2601137

Cited by 211 publications

(201 citation statements)

References 44 publications

Supporting

Mentioning

200

Contrasting

Order By: Relevance

“…Model-based methods detect motions by comparing the target with a built model. It is ideal to directly use the background image 22 without interference as the model if the scenario is static, but more often, using an estimated model from a priori knowledge is more actual, e.g., Gaussian mixture model (GMM) 23 is proposed for dynamic model estimation according to the Gaussian mixture distribution of pixels, which is widely applied for object tracking. In a very recent work, Haines and Xiang 24 further used a Dirichlet process GMM to provide a per-pixel density estimate for background computation.…”

Section: Motion Detectionmentioning

confidence: 99%

Energy flow: image correspondence approximation for motion analysis

Wang

Fang

2016

Opt. Eng

View full text Add to dashboard Cite

Abstract. We propose a correspondence approximation approach between temporally adjacent frames for motion analysis. First, energy map is established to represent image spatial features on multiple scales using Gaussian convolution. On this basis, energy flow at each layer is estimated using Gauss-Seidel iteration according to the energy invariance constraint. More specifically, at the core of energy invariance constraint is "energy conservation law" assuming that the spatial energy distribution of an image does not change significantly with time. Finally, energy flow field at different layers is reconstructed by considering different smoothness degrees. Due to the multiresolution origin and energy-based implementation, our algorithm is able to quickly address correspondence searching issues in spite of background noise or illumination variation. We apply our correspondence approximation method to motion analysis, and experimental results demonstrate its applicability. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

show abstract

Section: Motion Detectionmentioning

confidence: 99%

Energy flow: image correspondence approximation for motion analysis

Wang

Fang

2016

Opt. Eng

View full text Add to dashboard Cite

show abstract

“…The algorithm outputs an image whose style matches that of the example image. Shih et al [2014] spatially matches a casual portrait to a target example automatically found in a database; at its core, this technique relies on computationally expensive dense correspondence. Because of correspondence estimation and database dependence, this algorithm is a good candidate for cloud processing.…”

Section: Photo Editing and Photoshop Actionsmentioning

confidence: 99%

“…This includes content-aware photo enhancement [Kaufman et al 2012], dehazing [Kim et al 2013], edge-preserving enhancements Farbman et al 2008], colorization [Levin et al 2004], style transfer [Shih et al 2014;Aubry et al 2014], time-of-day hallucination [Shih et al 2013] but excludes dramatic changes such as inpainting. For this class of enhancements, the input and output images are usually highly correlated.…”

Section: Introductionmentioning

confidence: 99%

Transform recipes for efficient cloud photo enhancement

Gharbi¹,

Shih²,

Chaurasia³

et al. 2015

ACM Trans. Graph.

Self Cite

View full text Add to dashboard Cite

Figure 1: Cloud computing is often thought as an ideal solution to enable complex algorithms on mobile devices; by exploiting remote resources, one expects to reduce running time and energy consumption. However, this ignores the cost of transferring data over the network, the overhead of which often negates the benefits of the cloud, especially for data-heavy image processing applications. We introduce a new image processing pipeline that reduces the amount of transmitted data. The core of our approach is the transform recipe, a representation of the image transformation applied to a photograph that is compact and can be accurately estimated from an aggressively compressed input photograph. These properties make cloud computing more efficient in situations where transferring standard JPEG-compressed images would be too costly in terms of energy and/or time. In the example above, where we used an aggressive setting, our approach reduces the amount of transmitted data up to 80× compared to standard JPEG compressed images. It produces an output visually similar to the ground-truth result computed directly from the original uncompressed photograph. Photograph from the MIT 5k dataset [2011]. AbstractCloud image processing is often proposed as a solution to the limited computing power and battery life of mobile devices: it allows complex algorithms to run on powerful servers with virtually unlimited energy supply. Unfortunately, this overlooks the time and energy cost of uploading the input and downloading the output images. When transfer overhead is accounted for, processing images on a remote server becomes less attractive and many applications do not benefit from cloud offloading. We aim to change this in the case of image enhancements that preserve the overall content of an image. Our key insight is that, in this case, the server can compute and transmit a description of the transformation from input to output, which we call a transform recipe. At equivalent quality, our recipes are much more compact than JPEG images: this reduces the client's download. Furthermore, recipes can be computed from highly compressed inputs which significantly reduces the data uploaded to the server. The client reconstructs a high-fidelity approximation of the output by applying the recipe to its local high-quality input. We demonstrate our results on 168 images and 10 image processing applications, showing that our recipes form a compact representation for a diverse set of image filters. With an equivalent transmission budget, they provide higher-quality results than JPEG-compressed input/output images, with a gain of the order of 10 dB in many cases. We demonstrate the utility of recipes on a mobile phone by profiling the energy consumption and latency for both local and cloud computation: a transform recipe-based pipeline runs 2-4× faster and uses 2-7× less energy than local or naive cloud computation.

show abstract

“…For this, we establish correspondences among silhouettes and other relevant edges visible in both O, and in P o -the perspective projection image of P as seen from the same viewpoint used for image-model registration. Such correspondences are used to create a coherent feature-correspondence mapping (similar to the dense correspondence in Shih et al's work [33]). The silhouette edges of O and P o are automatically retrieved using the tech- nique by Suzuki and Abe [34].…”

Section: Feature-correspondence Mappingmentioning

confidence: 99%

Artistic relighting of paintings and drawings

Henz

Oliveira

2015

Vis Comput

View full text Add to dashboard Cite

We present a practical solution to the problem of subject relighting in paintings and drawings. Our interactive technique uses 3-D shading proxies and can be applied to objects with arbitrary geometries. Given a user-provided guess for the shading of an object in a painting/drawing and its corresponding target shading, we refine them using shading-color correlation and a multi-scale scheme. These refined shadings are then used to create a multi-channel shading-ratio image to perform relighting, while taking into account the colors used by the artists to convey shading information. We demonstrate the effectiveness of our solution on a variety of artistic styles, including paintings with strong brush strokes and unconventional shading encodings, drawings, and other types of artwork. Our method is the first to perform relighting of paintings and drawings and, in addition to relighting, can transfer smooth normal and depth maps from 3-D proxies to images.

show abstract

Style transfer for headshot portraits

Cited by 211 publications

References 44 publications

Energy flow: image correspondence approximation for motion analysis

Energy flow: image correspondence approximation for motion analysis

Transform recipes for efficient cloud photo enhancement

Artistic relighting of paintings and drawings

Contact Info

Product

Resources

About