Deepcomics

Bannier, Kévin; Jain, Eakta; Meur, Olivier Le

doi:10.1145/3204493.3204560

Cited by 7 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the DeepComics saliency dataset [14], we found that the segmentations of our Image-Overlaid webpage cluster closely resemble that of comics images. We applied the same model trained on WebSaliency for saliency detection in comics.…”

Section: A Saliency In Comicsmentioning

confidence: 81%

“…Further along [13], [35] investigated saliency in mobile UIs. Bannier et al [14] applied deep networks in saliency prediction on comics. Recently, [10] presented a model that first classifies a graphic design and then predicts design-specific importance.…”

Section: B Attention Prediction On Graphic Designsmentioning

confidence: 99%

“…We used the same trained model on our WebSaliency dataset for saliency prediction of comic images. Detailed cluster analysis wasn't feasible due to the limited collection of comic images (23 only) [14]. Therefore, we used our model without layout representation to obtain the predictions.…”

Section: A Saliency In Comicsmentioning

confidence: 99%

“…Fig.1. Overlaid predicted fixation density maps of the SAM-ResNet model and our AGD model (trained on our WebSaliency dataset) on the ieee.org webpage (from FiWI[2]), a mobile UI image (from Mobile UI[13]), a comics image (from DeepComics[14]) and a natural image (from MIT 1003[15]) and scanpaths for the webpage and the mobile UI instance. Our predictions more closely resembles the ground truth and our model shows great generalization across different types of graphic designs although the model is trained to predict fixations on webpage images.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Predicting Visual Attention in Graphic Design Documents

Chakraborty

Wei

Kelton

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

We present a model for predicting visual attention during the free viewing of graphic design documents. While existing works on this topic have aimed at predicting static saliency of graphic designs, our work is the first attempt to predict both spatial attention and dynamic temporal order in which the document regions are fixated by gaze using a deep learning based model. We propose a two-stage model for predicting dynamic attention on such documents, with webpages being our primary choice of document design for demonstration. In the first stage, we predict the saliency maps for each of the document components (e.g. logos, banners, texts, etc. for webpages) conditioned on the type of document layout. These component saliency maps are then jointly used to predict the overall document saliency. In the second stage, we use these layout-specific component saliency maps as the state representation for an inverse reinforcement learning model of fixation scanpath prediction during document viewing. To test our model, we collected a new dataset consisting of eye movements from 41 people freely viewing 450 webpages (the largest dataset of its kind). Experimental results show that our model outperforms existing models in both saliency and scanpath prediction for webpages, and also generalizes very well to other graphic design documents such as comics, posters, mobile UIs, etc. and natural images.

show abstract

Section: A Saliency In Comicsmentioning

confidence: 81%

Section: B Attention Prediction On Graphic Designsmentioning

confidence: 99%

Section: A Saliency In Comicsmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Predicting Visual Attention in Graphic Design Documents

Chakraborty

Wei

Kelton

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

Parsing and Summarizing Infographics with Synthetically Trained Icon Detection

Madan

Bylinskii

Nobre

et al. 2021

2021 IEEE 14th Pacific Visualization Symposium (PacificVis)

View full text Add to dashboard Cite

Figure 1: Infographics effectively convey messages using a mix of text and pictures (a). Automatically parsing infographics is challenging due to abstract concepts and diverse visual styles. We present an approach to detect and tag icons in infographics (see Fig. 8 for final results). We accomplish this by curating and annotating a dataset of infographics called Visually29K (b), and training an icon detection model on synthetic data generated by pasting icons onto patches of infographics (c). We also present three demo applications: topic prediction, multi-modal summarization -outputing text tags and visual hashtags representative of an infographic's topics (d), and multi-modal search -prioritizing infographics containing both visual and textual elements matching a query. ©oceanservice.noaa.gov, Evanmade Graphic Design, Walker Sands, Overson Roofing.

show abstract

Real-Time Multi-Map Saliency-Driven Gaze Behavior for Non-Conversational Characters

Goudé

Bruckert

Olivier

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.

show abstract

Deepcomics

Cited by 7 publications

References 24 publications

Predicting Visual Attention in Graphic Design Documents

Predicting Visual Attention in Graphic Design Documents

Parsing and Summarizing Infographics with Synthetically Trained Icon Detection

Real-Time Multi-Map Saliency-Driven Gaze Behavior for Non-Conversational Characters

Contact Info

Product

Resources

About