Challenging deep learning models with image distortion based on the abutting grating illusion

Fan, Jie; Zeng, Yi

doi:10.1016/j.patter.2023.100695

Cited by 2 publications

(1 citation statement)

References 93 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies have significantly advanced our ability to examine visual illusions by providing systematic data and tools. These efforts include the introduction of tools for calculating and generating illusory images systematically (Hirsch and Tal, 2020;Fan and Zeng, 2023), the development of open-source software with a parametric framework for controlled illusion generation (Makowski et al, 2021), and the proposal of a framework synthesizing new visual illusions using automatic differentiation techniques (Gomez-Villa et al, 2022). With the goal of evaluating machine visual illusions, prior research (Gomez-Villa et al, 2019, 2020Afifi and Brown, 2019;Benjamin et al, 2019) has also demonstrated that convolutional neural networks trained on ImageNet or low-level vision tasks can be misled by certain visual illusions, similar to human responses.…”

Section: Machine Visual Illusionmentioning

confidence: 99%

Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?

Zhang,

Pan,

Zhou

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Vision-Language Models (VLMs) are trained on vast amounts of data captured by humans emulating our understanding of the world. However, known as visual illusions, human's perception of reality isn't always faithful to the physical world. This raises a key question: do VLMs have the similar kind of illusions as humans do, or do they faithfully learn to represent reality? To investigate this question, we build a dataset containing five types of visual illusions and formulate four tasks to examine visual illusions in state-of-the-art VLMs. Our findings have shown that although the overall alignment is low, larger models are closer to human perception and more susceptible to visual illusions. Our dataset and initial findings will promote a better understanding of visual illusions in humans and machines and provide a stepping stone for future computational models that can better align humans and machines in perceiving and communicating about the shared visual world. The code and data are available at github.com/vl-illusion/dataset.

show abstract

Section: Machine Visual Illusionmentioning

confidence: 99%

Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?

Zhang,

Pan,

Zhou

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

The Quest for an Integrated Set of Neural Mechanisms Underlying Object Recognition in Primates

Kar,

DiCarlo

2024

Annual Review of Vision Science

View full text Add to dashboard Cite

Inferences made about objects via vision, such as rapid and accurate categorization, are core to primate cognition despite the algorithmic challenge posed by varying viewpoints and scenes. Until recently, the brain mechanisms that support these capabilities were deeply mysterious. However, over the past decade, this scientific mystery has been illuminated by the discovery and development of brain-inspired, image-computable, artificial neural network (ANN) systems that rival primates in these behavioral feats. Apart from fundamentally changing the landscape of artificial intelligence, modified versions of these ANN systems are the current leading scientific hypotheses of an integrated set of mechanisms in the primate ventral visual stream that support core object recognition. What separates brain-mapped versions of these systems from prior conceptual models is that they are sensory computable, mechanistic, anatomically referenced, and testable (SMART). In this article, we review and provide perspective on the brain mechanisms addressed by the current leading SMART models. We review their empirical brain and behavioral alignment successes and failures, discuss the next frontiers for an even more accurate mechanistic understanding, and outline the likely applications.

show abstract

Challenging deep learning models with image distortion based on the abutting grating illusion

Cited by 2 publications

References 93 publications

Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?

Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?

The Quest for an Integrated Set of Neural Mechanisms Underlying Object Recognition in Primates

Contact Info

Product

Resources

About