Fairness Indicators for Systematic Assessments of Visual Feature Extractors

Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision

“…We full-finetune all models on same subset of ImageNet-22K dataset. We then, for each gender and skintone perform inference of transferred models on the Casual Conversations Dataset and measure percentage of images associated with different labels at confidence threshold 0.1 following [51]. We observe that our model makes the least Harmful predictions and most Human predictions on images of people.…”

“…Recently, effort has been made by Yang et al [136] to reduce these biases by removing 2,702 synsets (out of 2,800 total) from the person subtree used in ImageNet. Motivated by the importance of building socially responsible models, we follow recent works [51] to systematically study the fairness, harms and biases of our models trained using self-supervised learning on random group of internet images.…”

“…We follow the protocols et al [51] to probe the performance of our larger SEER models on three different fair- We note that our motivation behind these fairness probes is not to validate the use of any given model. As noted in [51], for a given model, the choice of what fairness probes to measure depends on the application and use context.…”

“…We follow the protocols et al [51] to probe the performance of our larger SEER models on three different fair- We note that our motivation behind these fairness probes is not to validate the use of any given model. As noted in [51], for a given model, the choice of what fairness probes to measure depends on the application and use context. This choice must be thoroughly assessed by the stakeholder so as to answer why those probes are chosen, what kind of assumptions are embedded in this choice, and what specific questions do the system designers aim to answer [67,75].…”

“…We directly apply the benchmark protocol (including data preparation) as proposed in [51]. In this experiment, we perform similarity search, which requires a set of Queries and a Database.…”