Saliency Prediction via Multi-Level Features and Deep Supervision for Children with Autism Spectrum Disorder

Wei, Weijie; Li, Zhi; Huang, Lijin; Nebout, Alexis; Meur, Olivier Le

doi:10.1109/icmew.2019.00119

Cited by 22 publications

(23 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this paper, we focus on our ability to simulate the visual deployment of ASD people. Several attempts have been proposed recently such as [7]- [9], [12]. We however question the appropriateness of modelling the visual behavior of ASD people.…”

Section: Discussionmentioning

confidence: 99%

“…There is currently a growing interest in determining the visual factors that attract or repel the visual attention of ASD people. Thanks to new eye-tracking datasets involving ASD people [6], the number of saliency models is significantly increasing [7]- [9]. All these models aim to output a 2D saliency map from an input image.…”

Section: Introductionmentioning

confidence: 99%

“…As indicated previously, the most recent and performant saliency models rely on deep-based architectures [7]- [9], [12]. The training procedure is generally composed of two steps in order to deal with the lack of eye-tracking data of ASD people.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

From Kanner Austim to Asperger Syndromes, the Difficult Task to Predict Where ASD People Look at

et al. 2020

Self Cite

View full text Add to dashboard Cite

Modelling the visual attention of people with autism spectrum disorder (ASD) is attracting more and more interest. This consists in determining where ASD people look and in inferring the important visual features contributing to the gaze deployment. In this paper, we investigate whether or not existing neurotypical as well as ASD saliency models perform well over the whole spectrum of autism. For this purpose, we propose two new eye-tracking datasets of ASD people in order to cover a large part of the autism spectrum, going from high-level functioning (e.g. Asperger) to low-level functioning (e.g. Kanner) autism. We demonstrate that current neurotypical and ASD models do not generalize well and perform well only on a small part of the spectrum. Our objective is to raise the awareness of computer scientists to the difficult task we are facing up when it comes to simulate the gaze deployment of ASD people.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

From Kanner Austim to Asperger Syndromes, the Difficult Task to Predict Where ASD People Look at

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other recent studies have focused on predicting the visual attention of children with ASD. For instance, Wei et al [25] proposed a saliency prediction model based on a convolutional neural network (CNN), but they concluded that it is necessary to first train the model on an eye-tracking data set of typical development to enable more effective saliency prediction. Jiang et al [26] proposed a method with 86% accuracy that classifies eye fixations based on a comprehensive set of features and that integrates task performance, gaze information, and facial features extracted using a deep neural network.…”

Section: For Asd Screeningmentioning

confidence: 99%

Computer-Aided Screening of Autism Spectrum Disorder: Eye-Tracking Study Using Data Visualization and Deep Learning

Cilia¹,

Carette²,

Elbattah³

et al. 2021

JMIR Hum Factors

View full text Add to dashboard Cite

Background The early diagnosis of autism spectrum disorder (ASD) is highly desirable but remains a challenging task, which requires a set of cognitive tests and hours of clinical examinations. In addition, variations of such symptoms exist, which can make the identification of ASD even more difficult. Although diagnosis tests are largely developed by experts, they are still subject to human bias. In this respect, computer-assisted technologies can play a key role in supporting the screening process. Objective This paper follows on the path of using eye tracking as an integrated part of screening assessment in ASD based on the characteristic elements of the eye gaze. This study adds to the mounting efforts in using eye tracking technology to support the process of ASD screening Methods The proposed approach basically aims to integrate eye tracking with visualization and machine learning. A group of 59 school-aged participants took part in the study. The participants were invited to watch a set of age-appropriate photographs and videos related to social cognition. Initially, eye-tracking scanpaths were transformed into a visual representation as a set of images. Subsequently, a convolutional neural network was trained to perform the image classification task. Results The experimental results demonstrated that the visual representation could simplify the diagnostic task and also attained high accuracy. Specifically, the convolutional neural network model could achieve a promising classification accuracy. This largely suggests that visualizations could successfully encode the information of gaze motion and its underlying dynamics. Further, we explored possible correlations between the autism severity and the dynamics of eye movement based on the maximal information coefficient. The findings primarily show that the combination of eye tracking, visualization, and machine learning have strong potential in developing an objective tool to assist in the screening of ASD. Conclusions Broadly speaking, the approach we propose could be transferable to screening for other disorders, particularly neurodevelopmental disorders.

show abstract

“…Jia et al [19] proposed a multimodal salient wave detection network for sleep staging called SalientSleepNet, which translated the time series classification problem into a saliency detection problem and applies it to sleep stage classification. Wei et al [125] used a saliency model to pursue their research on autism spectrum disorder (ASD). They found that children with ASD, particularly autism, were informed by special objects and less on social objects (e.g., face), and the application of the verification model of obviousness is helpful in monitoring and evaluating their condition.…”

mentioning

confidence: 99%

Review of Visual Saliency Prediction: Development Process from Neurobiological Basis to Deep Models

et al. 2021

View full text Add to dashboard Cite

The human attention mechanism can be understood and simulated by closely associating the saliency prediction task to neuroscience and psychology. Furthermore, saliency prediction is widely used in computer vision and interdisciplinary subjects. In recent years, with the rapid development of deep learning, deep models have made amazing achievements in saliency prediction. Deep learning models can automatically learn features, thus solving many drawbacks of the classic models, such as handcrafted features and task settings, among others. Nevertheless, the deep models still have some limitations, for example in tasks involving multi-modality and semantic understanding. This study focuses on summarizing the relevant achievements in the field of saliency prediction, including the early neurological and psychological mechanisms and the guiding role of classic models, followed by the development process and data comparison of classic and deep saliency prediction models. This study also discusses the relationship between the model and human vision, as well as the factors that cause the semantic gaps, the influences of attention in cognitive research, the limitations of the saliency model, and the emerging applications, to provide new saliency predictions for follow-up work and the necessary help and advice.

show abstract

Saliency Prediction via Multi-Level Features and Deep Supervision for Children with Autism Spectrum Disorder

Cited by 22 publications

References 15 publications

From Kanner Austim to Asperger Syndromes, the Difficult Task to Predict Where ASD People Look at

From Kanner Austim to Asperger Syndromes, the Difficult Task to Predict Where ASD People Look at

Computer-Aided Screening of Autism Spectrum Disorder: Eye-Tracking Study Using Data Visualization and Deep Learning

Review of Visual Saliency Prediction: Development Process from Neurobiological Basis to Deep Models

Contact Info

Product

Resources

About