Scalable and coherent video resizing with per-frame optimization

Wang, Yu-Shuen; Hsiao, Jen-Hung; Sorkine, Olga; Lee, Tong‐Yee

doi:10.1145/1964921.1964983

Cited by 17 publications

(19 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…8: Without the temporal smoothness term, the similarity transform parameters of faces jitters (blue curves). Adding the temporal smoothness term in (12) slightly alleviates the instability (green curves). With the coherent embedding term in (13), the similarity transform becomes temporally stable (red curves).…”

Section: Mesh Optimizationmentioning

confidence: 99%

Correcting Face Distortion in Wide-Angle Videos

Lai,

Shih,

Liang

et al. 2021

Preprint

View full text Add to dashboard Cite

Video blogs and selfies are popular social media formats, which are often captured by wide-angle cameras to show human subjects and expanded background. Unfortunately, due to perspective projection, faces near corners and edges exhibit apparent distortions that stretch and squish the facial features, resulting in poor video quality. In this work, we present a video warping algorithm to correct these distortions.Our key idea is to apply stereographic projection locally on the facial regions. We formulate a mesh warp problem using spatial-temporal energy minimization and minimize background deformation using a line-preservation term to maintain the straight edges in the background. To address temporal coherency, we constrain the temporal smoothness on the warping meshes and facial trajectories through the latent variables. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths. The user study shows that 83.9% of users prefer our algorithm over other alternatives based on perspective projection. The video results can be found at https://www.wslai.net/publications/video_face_correction/.

show abstract

Section: Mesh Optimizationmentioning

confidence: 99%

Correcting Face Distortion in Wide-Angle Videos

Lai,

Shih,

Liang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Content-aware video re-targeting techniques [19], [20] require importance maps to preserve visual information while solving optimization problems. The importance maps are generally based on a combination of low-level (e.g., image gradient and optical flow) and high-level cues (e.g., visual saliency [21] and detected faces).…”

Section: Visual Saliency and Semanticsmentioning

confidence: 99%

Semantic-Driven Generation of Hyperlapse from 360 Degree Video

Lai

Huang

Joshi

et al. 2018

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Abstract-We present a system for converting a fully panoramic (360 • ) video into a normal field-of-view (NFOV) hyperlapse for an optimal viewing experience. Our system exploits visual saliency and semantics to non-uniformly sample in space and time for generating hyperlapses. In addition, users can optionally choose objects of interest for customizing the hyperlapses. We first stabilize an input 360 • video by smoothing the rotation between adjacent frames and then compute regions of interest and saliency scores. An initial hyperlapse is generated by optimizing the saliency and motion smoothness followed by the saliency-aware frame selection. We further smooth the result using an efficient 2D video stabilization approach that adaptively selects the motion model to generate the final hyperlapse. We validate the design of our system by showing results for a variety of scenes and comparing against the state-of-the-art method through a large-scale user study.

show abstract

“…Importance-based scaling (also called "warping") is another popular group of retargeting approaches [18], [19]. Instead of cutting out important content, the frames are divided into smaller image areas which are then scaled down separately according to their respective importance.…”

Section: Literature Reviewmentioning

confidence: 99%

sZoom: A Framework for Automatic Zoom into High Resolution Surveillance Videos

Saini¹,

Guthier²,

Kuang³

et al. 2019

Preprint

View full text Add to dashboard Cite

Current cameras are capable of recording high resolution video. While viewing on a mobile device, a user can manually zoom into this high resolution video to get more detailed view of objects and activities. However, manual zooming is not suitable for surveillance and monitoring. It is tiring to continuously keep zooming into various regions of the video. Also, while viewing one region, the operator may miss activities in other regions. In this paper, we propose sZoom, a framework to automatically zoom into a high resolution surveillance video. The proposed framework selectively zooms into the sensitive regions of the video to present details of the scene, while still preserving the overall context required for situation assessment. A multi-variate Gaussian penalty is introduced to ensure full coverage of the scene. The method achieves near real-time performance through a number of timing optimizations. An extensive user study shows that, while watching a full HD video on a mobile device, the system enhances the security operator's efficiency in understanding the details of the scene by 99% on the average compared to a scaled version of the original high resolution video. The produced video achieved 46% higher ratings for usefulness in a surveillance task.

show abstract

Scalable and coherent video resizing with per-frame optimization

Cited by 17 publications

References 35 publications

Correcting Face Distortion in Wide-Angle Videos

Correcting Face Distortion in Wide-Angle Videos

Semantic-Driven Generation of Hyperlapse from 360 Degree Video

sZoom: A Framework for Automatic Zoom into High Resolution Surveillance Videos

Contact Info

Product

Resources

About