Understanding Visual Saliency in Mobile User Interfaces

Leiva, Luis A.; Xue, Yunfei; Bansal, Anupma; Tavakoli, Hamed R.; Köroðlu, Tuðçe; Du, Jingzhou; Dayama, Niraj Ramesh; Oulasvirta, Antti

doi:10.1145/3379503.3403557

Cited by 18 publications

(20 citation statements)

References 55 publications

(102 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saliency models are not only useful to model human visual attention on natural scenes but also more broadly applicable, such as to information visualisations [14], web pages [5,33], mobile user interfaces [15,34], or graphical user interfaces [19]. An increasing number of works have explored attention models in the context of information visualisations [14,35].…”

Section: Computational Modelling Of Visual Attentionmentioning

confidence: 99%

Scanpath Prediction on Information Visualisations

Wang

Bâce

Bulling

2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

We propose Unified Model of Saliency and Scanpaths (UMSS)-a model that learns to predict multi-duration saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect to several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5 % for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6 % for saliency prediction. These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.

show abstract

Section: Computational Modelling Of Visual Attentionmentioning

confidence: 99%

Scanpath Prediction on Information Visualisations

Wang

Bâce

Bulling

2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…Saliency models are not only useful to model human visual attention on natural scenes but also more broadly applicable, such as to information visualisations [14], web pages [31,32], mobile user interfaces [15,33], or graphical user interfaces [19]. An increasing number of works have explored attention models in the context of information visualisations [5].…”

Section: Computational Modelling Of Visual Attentionmentioning

confidence: 99%

Scanpath Prediction on Information Visualisations

Wang¹,

Bâce²,

Bulling³

2021

Preprint

View full text Add to dashboard Cite

We propose Unified Model of Saliency and Scanpaths (UMSS)-a model that learns to predict visual saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect to several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5 % for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6 % for saliency prediction. These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.

show abstract

“…Saliency models can predict either (static) saliency maps [41] or (dynamic) scanpaths [19]. Most research has focused on predicting saliency maps [2,9,15,17,20], overlooking key temporal aspects like fixation timing and duration. Saliency maps show aggregated fixation locations, i.e., areas that users will pay attention to, where the eye remains relatively static.…”

Section: Introductionmentioning

confidence: 99%

“…We use DeepGaze++ [17] as a reference model to investigate the potential impact that different design parameters may have in scanpath prediction. DeepGaze++ is a state-of-the-art scanpath model for visual saliency prediction that has shown promising results.…”

Section: Introductionmentioning

confidence: 99%

Impact of Design Decisions in Scanpath Modeling

Emami,

Jiang,

Guo

et al. 2024

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

Modeling visual saliency in graphical user interfaces (GUIs) allows to understand how people perceive GUI designs and what elements attract their attention. One aspect that is often overlooked is the fact that computational models depend on a series of design parameters that are not straightforward to decide. We systematically analyze how different design parameters affect scanpath evaluation metrics using a state-of-the-art computational model (DeepGaze++). We particularly focus on three design parameters: input image size, inhibition-of-return decay, and masking radius. We show that even small variations of these design parameters have a noticeable impact on standard evaluation metrics such as DTW or Eyenalysis. These effects also occur in other scanpath models, such as UMSS and ScanGAN, and in other datasets such as MASSVIS. Taken together, our results put forward the impact of design decisions for predicting users' viewing behavior on GUIs.

show abstract

Understanding Visual Saliency in Mobile User Interfaces

Cited by 18 publications

References 55 publications

Scanpath Prediction on Information Visualisations

Scanpath Prediction on Information Visualisations

Scanpath Prediction on Information Visualisations

Impact of Design Decisions in Scanpath Modeling

Contact Info

Product

Resources

About