The Unreasonable Ineptitude of Deep Image Classification Networks

Abstract

The success of deep image classification networks has been met with enthusiasm and investment from both the academic community and industry. We hypothesize users will expect these systems to behave similarly to humans, and to succeed and fail in ways humans do. To investigate this, we tested six popular image classifiers on imagery from ten tool categories, examining how 17 visual transforms impacted both human and AI classification. Results showed that (1) none of the visual transforms we examined produced substantial impairment for human recognition; (2) human errors were limited to mostly to functional confusions; (3) almost all visual transforms impacted nearly every image classifier negatively and often catastrophically; (4) human expectations about performance of AI classifiers map more closely onto human error than AI performance; and (5) models trained with an enriched training set involving examples of the transformed imagery achieved improved performance but were not inoculated from error.

Assessing Satisfaction in and Understanding of a Collaborative Explainable AI (Cxai) System through User Studies

“…In the two studies presented here, we report tests of comprehension and performance (in Study 1) of users interacting with explanations generated via CXAI, and qualitative assessments of satisfaction by human participant responses to the CXAI system or explanations generated by that tool (in Study 2). In both user studies, we compared the CXAI system (Mamun et al, 2021b) to a Visual Browser (Mueller et al, 2020) of an image classification database [see Figure 2] which enabled users to explore patterns and see the results of the image classifier. This visual browser was the same interface CXAI users had access to when creating CXAI entries.…”

Assessing Collaborative Explanations of AI using Explanation Goodness Criteria

“…The CXAI system 1 itself was developed using Laravel. To populate the system, we created an online browser that allowed users to explore how a popular commercial image classifier performed on a set of 50 images of ten hand tools under several image transforms (see Mueller et al, 2020, which examined the performance of the system). The overall system was developed collaboratively with a set of users including the design team and interested graduate students enrolled in a human factor graduate program as part of their coursework, who were asked to explore the AI system and use the CXAI system to identify errors, patterns, and other issues with the system.…”