SalyPath360: Saliency and scanpath prediction framework for omnidirectional images

Abstract

This paper introduces a new framework to predict the visual attention of omnidirectional images. The key setup of our architecture is the simultaneous prediction of the saliency map and a corresponding scanpath for a given stimulus. The framework implements a fully encoder-decoder convolutional neural network augmented by an attention module to generate representative saliency maps. In addition, an auxiliary network is employed to generate probable viewport center fixation points through the So f tArgMax function. The latter allows deriving fixation points from feature maps. To take advantage of the scanpath prediction, an adaptive joint probability distribution model is then applied to construct the final unbiased saliency map by leveraging the encoder decoder-based saliency map and the scanpathbased saliency heatmap. The proposed framework was evaluated in terms of saliency and scanpath prediction, and the results were compared to state-of-the-art methods on Salient360! dataset. The results showed the relevance of our framework and the benefits of such architecture for further omnidirectional visual attention prediction tasks.

Enhancing Immersive Experiences through 3D Point Cloud Analysis: A Novel Framework for Applying 2D Visual Saliency Models to 3D Point Clouds

“…A critical aspect of delivering superior immersive experiences involves accurately modeling the visual attentive behavior of users toward 3D point clouds within transmission systems [11]. Human observers inherently prioritize certain areas within their Field-of-View (FoV), concentrating on regions of interest while disregarding others [12], [13]. These selective processes so-called attention mechanisms enable individuals to efficiently interpret and comprehend complex scenes by allocating their limited perceptual and cognitive resources towards the most relevant segments of sensory input [14].…”