Apparent resolution enhancement for animations

Abstract

Figure 1: Highly detailed animations can be difficult to reproduce on current display devices. Here, four frames from animations used to evaluate our apparent resolution enhancement are shown: (a) rendering of a ball textured with text; (b) rendering of a detailed assembly of fibers; (c) a rendered short film ("Big Buck Bunny" c by Blender Foundation); (d) high-resolution content captured with a video camera. AbstractPresenting the variety of high resolution images captured by highquality devices, or generated on the computer, is challenging due to the limited resolution of current display devices. Our recent work addressed this problem by taking into account human perception. By applying a specific motion to a high-resolution image shown on a low-resolution display device, human eye tracking and integration could be exploited to achieve apparent resolution enhancement. To this end, the high-resolution image is decomposed into a sequence of temporally varying low-resolution images that are displayed at high refresh rates. However, this approach is limited to a specific class of simple or constant movements, i. e. "panning". In this work, we generalize this idea to arbitrary motions, as well as to videos with arbitrary motion flow. The resulting image sequences are compared to a range of other down-sampling methods.

Apparent resolution enhancement for motion videos

“…In such cases viewers typically scan the scene quickly and then track one object of interest in the frame [Boccignone et al 2002;Henderson 2003]. Hence, similarly to [Templin et al 2011;Boccignone et al 2002], we predict the eye movement j(t, x) at every pixel x of the input frame as the local motion of the video content and estimate it by computing a dense optical flow [Brox . Thus, we replace j(t) in (4) by j(t, x).…”

“…Unlike the previous method of Templin et al [2011], our perceived video model (5) allows us to use the precise display PSF function in order to achieve optimal performance. We demonstrate the importance of this consideration in Section 4.…”

“…At every instant, only half of the pixels are shown, but the viewer integrates this information temporally and perceives one continuous image. As we discussed in the Introduction, recent works in computer graphics also use the temporal and spatial integration properties of the eye to increase the resolution of displays spatially [Templin et al 2011;Didyk et al 2010a] as well as the temporal resolution of computer-generated videos [Didyk et al 2010b]. Berthouzoz and Fattal [2012] vibrate a display by a very small amplitude to obtain the sub-pixel offsets between several rapidly displayed low-resolution images.…”

“…The displayed images are designed to produce a perceived image with an apparent resolution that is higher than the display resolution. Templin et al [2011] extend this approach to animations by relying on the natural tendency of a viewer to track moving objects in the video. This tracking leads to the sub-pixel offsets needed for retinal resolution enhancement.…”

Resolution enhancement by vibrating displays

“…The same principle is used in digital light processing projectors that display the RGB color components sequentially and rely on the human eye to temporally merge the components. As we discussed in the Introduction, recent works in computer graphics also use the temporal and spatial integration properties of the eye to increase the resolution of displays spatially [Didyk et al 2010a] as well as the temporal and spatial resolution of computer-generated videos [Didyk et al 2010b;Templin et al 2011]. Our display system is inspired by this line of work.…”