Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images

Abstract

Depth adjustment aims to enhance the visual experience of stereoscopic 3D (S3D) images, which accompanied with improving visual comfort and depth perception. For a human expert, the depth adjustment procedure is a sequence of iterative decision making. The human expert iteratively adjusted the depth until he is satisfied with the both levels of visual comfort and the perceived depth. In this work, we present a novel deep reinforcement learning (DRL)-based approach for depth adjustment named VCA-RL (Visual Comfort Aware Reinforcement Learning) to explicitly model human sequential decision making in depth editing operations. We formulate the depth adjustment process as a Markov decision process where actions are defined as camera movement operations to control the distance between the left and right cameras. Our agent is trained based on the guidance of an objective visual comfort assessment metric to learn the optimal sequence of camera movement actions in terms of perceptual aspects in stereoscopic viewing. With extensive experiments and user studies, we show the effectiveness of our VCA-RL model on three different S3D databases.

Enhancing image resolution of confocal fluorescence microscopy with deep learning

“…Chen et al proposed a visual comfort aware-reinforcement learning (VCARL) method for depth adjustment of stereoscopic 3D images. This method aims to improve the visual quality and comfort of 3D images by learning a depth adjustment policy from human feedback [160]. These advancements offer various means of manipulating objects, adjusting depth, and generating novel views, ultimately enhancing the quality and realism of 3D scene synthesis and editing.…”

Siamese-Discriminant Deep Reinforcement Learning for Solving Jigsaw Puzzles with Large Eroded Gaps

“…Reinforcement Learning (RL) has witnessed a great number of successes in combinatorial optimization (CO) (Bengio, Lodi, and Prouvost 2021;Woo, Lee, and Cho 2022;Chen and Tian 2019;Zong et al 2022;Bai et al 2021), robotics (Tomar, Sathuluri, and Ravindran 2019), natural language processing (Li, Kiseleva, and De Rijke 2019) and computer vision (Kim et al 2021). Combination of RL and graph neural network also flourished in recent years (Vesselinova et al 2020).…”