ZoomShop: Depth‐Aware Editing of Photographic Composition

Liu, Sean J.; Agrawala, Maneesh; DiVerdi, Stephen; Hertzmann, Aaron

doi:10.1111/cgf.14458

Cited by 5 publications

(3 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Working from a single viewpoint limits the options for capturing these phenomena, and necessarily means that they are curvilinear perspectives. Later, I will discuss multiperspective techniques that offer more options for adjusting relative object scale (Badki et al, 2017;Liu et al, 2022).…”

Section: Contentmentioning

confidence: 99%

“…One use is to make distant objects larger, in order to create "natural perspective" imagery, as discussed earlier. Computational Zoom requires a collection of photographs to be taken at the same time; our ZoomShop method (Liu et al, 2022) works from a single photograph and generalizes Computational Zoom to allow for smoother transitions between regions (and, thus, amounts to "infinite multiperspective" in some cases). Multiperspective projections in computer graphics can simulate other nonrealistic artistic techniques, such as de Chirico paintings (Agrawala et al, 2000), Hockney's joiners (Zelnik-Manor and Perona, 2007), cubism (Collomosse and Hall, 2003), and Disney multiperspective background panoramas (Wood et al, 1997).…”

Section: Computational Multiperspective Collagementioning

confidence: 99%

See 1 more Smart Citation

Toward a Theory of Perspective Perception in Pictures

Hertzmann

2023

Preprint

View full text Add to dashboard Cite

This paper reviews perspective perception in pictures, and proposes hypotheses for 3D shape and space perception in pictures, in which eye fixations and foveal vision play a central role. Perspective describes how painters and cameras produce 2D picture arrangements from observations of 3D space. Many past theories and experimental studies focus solely on linear perspective. Yet, these theories fail to explain many important perceptual phenomena in perspective, including the effectiveness of other approaches to projection; few classical paintings strictly obey linear perspective, nor do the best distortion-avoidance techniques for wide-angle computational photography. This paper hypothesizes a two-stage process for 3D vision, in which shape is locally interpreted for a current eye fixation in a picture, according to foveal vision, and a global interpretation of a picture arises from spatial relationships between these regions. This framework could provide new insight for understanding pictorial perception across many types of depiction and suggests new studies to explore these hypotheses.

show abstract

Section: Contentmentioning

confidence: 99%

Section: Computational Multiperspective Collagementioning

confidence: 99%

Toward a Theory of Perspective Perception in Pictures

Hertzmann

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This technique was later developed into a plugin for Maya [31]. More recently, Liu et al [32] have shown a depth-based editing tool for layering, resizing and recompositing photographs to overcome some of the restrictions of linear perspective-based projections and produce images that match visual space more effectively.…”

Section: Alternative Nonlinear Approachesmentioning

confidence: 99%

The Art and Science of Displaying Visual Space

Pepperell¹

2022

lim

View full text Add to dashboard Cite

This paper considers the problem of how to display visual space naturalistically in image media. A long-standing solution is linear perspective projection, which is currently used in imaging technologies from cameras to 3D graphics renderers. Linear perspective has many strengths but also some significant weaknesses and over the centuries alternative techniques have been developed for creating more naturalistic images. Here we discuss the problem, its scientific background, and some of the approaches taken by artists and computer graphics researchers to find solutions. We briefly introduce our own approach, which is a form of nonlinear 3D geometry modelled on the perceptual structure of visual space and designed to work on standard displays. We conclude that perceptually modelled nonlinear approaches can make 3D imaging technology more naturalistic than methods based on linear perspective.

show abstract

Enhancing Diffusion Models with 3D Perspective Geometry Constraints

Upadhyay,

Zhang,

et al. 2023

ACM Trans. Graph.

View full text Add to dashboard Cite

While perspective is a well-studied topic in art, it is generally taken for granted in images. However, for the recent wave of high-quality image synthesis methods such as latent diffusion models, perspective accuracy is not an explicit requirement. Since these methods are capable of outputting a wide gamut of possible images, it is difficult for these synthesized images to adhere to the principles of linear perspective. We introduce a novel geometric constraint in the training process of generative models to enforce perspective accuracy. We show that outputs of models trained with this constraint both appear more realistic and improve performance of downstream models trained on generated images. Subjective human trials show that images generated with latent diffusion models trained with our constraint are preferred over images from the Stable Diffusion V2 model 70% of the time. SOTA monocular depth estimation models such as DPT and PixelFormer, fine-tuned on our images, outperform the original models trained on real images by up to 7.03% in RMSE and 19.3% in SqRel on the KITTI test set for zero-shot transfer.

show abstract

ZoomShop: Depth‐Aware Editing of Photographic Composition

Cited by 5 publications

References 47 publications

Toward a Theory of Perspective Perception in Pictures

Toward a Theory of Perspective Perception in Pictures

The Art and Science of Displaying Visual Space

Enhancing Diffusion Models with 3D Perspective Geometry Constraints

Contact Info

Product

Resources

About