A Blind Stereoscopic Image Quality Evaluator With Segmented Stacked Autoencoders Considering the Whole Visual Perception Route

Abstract

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA. Index Terms-stereoscopic image quality assessment, retinal ganglion cell, lateral geniculate nucleus, segmented stacked autoencoders, edge quality, color quality. I. INTRODUCTION 3 D visual content has penetrated our lives deeply. We can easily find 3D movies, 3D TVs, 3D digital cameras and mobile phones equipped with dual cameras around us. Through them countless stereoscopic images are produced everyday. These images often suffer from perceptual quality degradation caused by distortions when they are transmitted, stored, compressed and processed. The degraded images need to be restored and SIQA indices can provide a criterion for restoration [1], [2]. Image quality assessment (IQA) models are divided into three categories according to usage of the original image: full-reference (FR) [3]-[12], reduced-reference

Unsupervised Saliency Detection of Rail Surface Defects using Stereoscopic Images

“…According to the principle of the triangulation and polar line correction, as shown in Fig. 2, the disparity map of the left and right cameras can be obtained by the stereo matching [35], [36]. Then, the corresponding depth map can be calculated based on the disparity map by combining the internal and external parameters of the cameras of BSC, as shown in Fig.…”

Blind Stereoscopic Image Quality Assessment Accounting for Human Monocular Visual Properties and Binocular Interactions

“…At the physiological level, it proved that V1 neurons receive the multiplexed signals from the summation and difference channels, so that they can tune to different disparities [50]. Besides, the summation and difference signals have been applied to evaluate the stereoscopic image quality and achieved better performance than previous models [43], [51]. Considering the binocular difference channel is missing from the single-channel model, we take the double-channel model to simulate human binocular interaction behavior.…”

“…After extracting all the above features, the LIBSVM package is utilized to solve the quality prediction problem [70]. In this paper, we adopt SVR with a radial basis function (RBF) kernel to train the prediction function, which is effectively used in other NR image quality assessment models [16], [43], [51].…”

MFNN: Position and Attitude Measurement Neural Network Based on Multi-Feature Fusion

“…We mainly study the latter. We use a single RGB image to measure the position and attitude of the object [20]. It achieves a great breakthrough in the calculation of object position and posture, and achieves good results.…”