Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines

Katti, Harish; Arun, S. P.

doi:10.1167/19.7.1

Cited by 10 publications

(5 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The stimuli comprised 20 co-registered Indian faces (19 male, 1 female) from the IISc Indian Face Dataset 5 . All faces were grayscale, upright and front-facing.…”

Section: Methodsmentioning

confidence: 99%

“…Convolutional or deep neural networks have revolutionized computer vision with their human-like accuracy on object-recognition tasks, and their object representations match coarsely with the brain 2 , 3 . Yet they are still outperformed by humans 4 , 5 and show systematic finer-scale deviations from human perception 6 – 9 . Even these differences are largely quantitative in that there are no explicit or emergent properties that are present in humans but absent in deep networks.…”

Section: Introductionmentioning

confidence: 98%

See 1 more Smart Citation

Qualitative similarities and differences in visual object representations between brains and deep networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

Deep neural networks have revolutionized computer vision, and their object representations across layers match coarsely with visual cortical areas in the brain. However, whether these representations exhibit qualitative patterns seen in human perception or brain representations remains unresolved. Here, we recast well-known perceptual and neural phenomena in terms of distance comparisons, and ask whether they are present in feedforward deep neural networks trained for object recognition. Some phenomena were present in randomly initialized networks, such as the global advantage effect, sparseness, and relative size. Many others were present after object recognition training, such as the Thatcher effect, mirror confusion, Weber’s law, relative size, multiple object normalization and correlated sparseness. Yet other phenomena were absent in trained networks, such as 3D shape processing, surface invariance, occlusion, natural parts and the global advantage. These findings indicate sufficient conditions for the emergence of these phenomena in brains and deep networks, and offer clues to the properties that could be incorporated to improve deep networks.

show abstract

“…The stimuli comprised 20 co-registered Indian faces (19 male, 1 female) from the IISc Indian Face Dataset 5 . All faces were grayscale, upright and front-facing.…”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

Qualitative similarities and differences in visual object representations between brains and deep networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Does the face-identity-trained CNN not just match the overall performance of humans, but also use similar strategies to solve the identity matching task ( 32 , 33 )? To address this question, we performed an analysis of the errors being made by humans and CNNs and computed the trial-by-trial predictivity of the human behavioral choices by CNNs (see SI Appendix , Supplementary Note 3 for details).…”

Section: Resultsmentioning

confidence: 99%

Behavioral signatures of face perception emerge in deep neural networks optimized for face recognition

Dobs

Yuan

Martinez

et al. 2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Human face recognition is highly accurate and exhibits a number of distinctive and well-documented behavioral “signatures” such as the use of a characteristic representational space, the disproportionate performance cost when stimuli are presented upside down, and the drop in accuracy for faces from races the participant is less familiar with. These and other phenomena have long been taken as evidence that face recognition is “special”. But why does human face perception exhibit these properties in the first place? Here, we use deep convolutional neural networks (CNNs) to test the hypothesis that all of these signatures of human face perception result from optimization for the task of face recognition. Indeed, as predicted by this hypothesis, these phenomena are all found in CNNs trained on face recognition, but not in CNNs trained on object recognition, even when additionally trained to detect faces while matching the amount of face experience. To test whether these signatures are in principle specific to faces, we optimized a CNN on car discrimination and tested it on upright and inverted car images. As we found for face perception, the car-trained network showed a drop in performance for inverted vs. upright cars. Similarly, CNNs trained on inverted faces produced an inverted face inversion effect. These findings show that the behavioral signatures of human face perception reflect and are well explained as the result of optimization for the task of face recognition, and that the nature of the computations underlying this task may not be so special after all.

show abstract

“…The first consists of technical studies that focus on the applications and capabilities of facial processing. This covers issues such as classification of faces based on regional affiliations -North and South Indian (Katti and Arun, 2019) or North, East and South Indian (Sarin and Panda, 2020), identification of genetic disorders in children (Narayanan et al, 2019), and detection of emotions (Singh and Benedict, 2020). The second stream of work consists of research papers, reports and other critical perspectives on the use of facial processing, highlighting associated risks, harms, and modes of regulation (Bhandari, 2021;Joshi, 2020;Aneja and Chamuah, 2020;Kovacs, 2020;Parsheera, 2019;Marda, 2019a;Basu and Sonkar, 2019).…”

Section: Related Workmentioning

confidence: 99%

Cinderella's shoe won't fit Soundarya: An audit of facial processing tools on Indian faces

Jain¹,

Parsheera²

2021

Preprint

View full text Add to dashboard Cite

The increasing adoption of facial processing systems in India is fraught with concerns of privacy, transparency, accountability, and missing procedural safeguards. At the same time, we also know very little about how these technologies perform on the diverse features, characteristics, and skin tones of India's 1.34 billion plus population. In this paper, we test the face detection and facial analysis functions of four commercial facial processing tools on a dataset of Indian faces. The tools display varying error rates in the face detection and gender and age classification functions. The gender classification error rate for Indian female faces is consistently higher compared to that of males -the highest female error rate being 14.68%. In some cases, this error rate is much higher than that shown by previous studies for females of other nationalities. Age classification errors are also high. Despite taking into account an acceptable error margin of plus or minus 10 years from a person's actual age, age prediction failures are in the range of 14.3% to 42.2%. These findings point to the limited accuracy of facial processing tools, particularly for certain demographic groups, and the need for more critical thinking before adopting such systems.

show abstract

Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines

Cited by 10 publications

References 47 publications

Qualitative similarities and differences in visual object representations between brains and deep networks

Qualitative similarities and differences in visual object representations between brains and deep networks

Behavioral signatures of face perception emerge in deep neural networks optimized for face recognition

Cinderella's shoe won't fit Soundarya: An audit of facial processing tools on Indian faces

Contact Info

Product

Resources

About