Computational rephotography

Abstract

Rephotographers aim to recapture an existing photograph from the same viewpoint. A historical photograph paired with a well-aligned modern rephotograph can serve as a remarkable visualization of the passage of time. However, the task of rephotography is tedious and often imprecise, because reproducing the viewpoint of the original photograph is challenging. The rephotographer must disambiguate between the six degrees of freedom of 3D translation and rotation, and the confounding similarity between the effects of camera zoom and dolly.We present a real-time estimation and visualization technique for rephotography that helps users reach a desired viewpoint during capture. The input to our technique is a reference image taken from the desired viewpoint. The user moves through the scene with a camera and follows our visualization to reach the desired viewpoint. We employ computer vision techniques to compute the relative viewpoint difference. We guide 3D movement using two 2D arrows. We demonstrate the success of our technique by rephotographing historical images and conducting user studies.

A transformation-aware perceptual image metric

“…Also, the concept of visual equivalence 1 can be extended to handle different scene views, as well as minor deformations of the object geometry and their relocation. Our quality metric could be applicable to rephotography and rerendering 3 allowing for a better judgement of structural image differences while ignoring minor misregistration problems. This is also the case in image retargeting, where all image distance metrics such as SIFT flow, bidirectional similarity, or earth mover's distance 4 account for some form of the energy cost needed to transform one image into another.…”

“…For image similarity or quality evaluation approaches, it is typically assumed the image pair is perfectly aligned (registered), which is directly granted in image compression, restoration, denoising, broadcasting, and rendering. However, in many other applications, such as visual equivalence judgement, 1 comparison of rendered and photographed scenes, 2 rephotography, 3 or image retargeting, 4 the similarity of images should be judged in the presence of distortions caused by transformations. Ecologically valid transformation 5 is a nonstructural distortion 6 and as such should be separated from others.…”

Interactive acquisition of residential floor plans

“…Instead of simply augmenting a user's view of the world, however, our projected output serves to guide an interactive reconstruction process. Directing the user in this way is similar to re-photography [10], where a user is guided to capture a photograph from the same viewpoint as in a previous photograph. By using a micro-projector as the output modality, our system allows the operator to focus on interacting with the environment.…”

Laser speckle photography for surface tampering detection

“…Later, the user comes back and needs to take a new speckle image from the same location to detect if the surface was touched. We first use vision-based rephotography [2,1] to guide the camera to a roughly accurate viewpoint within 3 mm. For flat surfaces where vision-based pose estimation is degenerate, we use structured light and the visual alignment of a checkerboard pattern instead.…”

“…While this tolerance is straightforward to achieve in laboratory settings, using carefully calibrated static cameras, for most practical applications of surface verification, the camera and laser often cannot remain fixed in front of the surface. To achieve high-precision viewpoint alignment, we use a combination of computational rephotography, a technique to guide camera to the desired viewpoint based on relative pose estimation [2], and a new analysis of speckle correlation. Our paper makes the following contributions:…”