Scalable Detection of Offensive and Non-compliant Content / Logo in Product Images

Gandhi, Shreyansh; Kokkula, Samrat; Chaudhuri, Anathbandhu; Magnani, Alessandro; Stanley, Theban; Ahmadi, Behzad; Kandaswamy, Venkatesh; Ovenc, Omer; Mannor, Shie

doi:10.1109/wacv45572.2020.9093454

Cited by 26 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the present paper, we focus on images following the definition (b). This definition aligns with definitions of previous work detecting hate speech [20] and offensive product images [16]. Note that inappropriateness, especially offensiveness, is a concept that is based on social norms, and people have diverse sentiments.…”

Section: Inappropriate Image Contentsupporting

confidence: 69%

“…Furthermore, they conducted a hand-surveyed image selection to identify misogynistic images in the ImageNet-ILSVRC-2012 (ImageNet1k) dataset. Gandhi et al [16] aimed to detect offensive product content using machine learning; however, they have described the lack of adequate training data. Recently, Nichol et al [33] applied CLIP to filter images of violent objects but also images portraying people and faces.…”

Section: Issues Arising From Large Datasetsmentioning

confidence: 99%

“…As baselines we fine-tuned two standard pre-trained CV models (PMs) to investigate how well deep neural networks can identify inappropriate content. Similar to Gandhi et al [16], we used the ResNet50 architecture [24], pre-trained on ImageNet datasets [9]. Fig.…”

Section: Inappropriate Content Detection Of Q16mentioning

confidence: 99%

See 2 more Smart Citations

Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?

Schramowski¹,

Tauchmann²,

Kersting³

2022

Preprint

View full text Add to dashboard Cite

This paper contains images and descriptions that are offensive in nature.Large datasets underlying much of current machine learning raise serious issues concerning inappropriate content such as offensive, insulting, threatening, or might otherwise cause anxiety. This calls for increased dataset documentation, e.g., using datasheets. They, among other topics, encourage to reflect on the composition of the datasets. So far, this documentation, however, is done manually and therefore can be tedious and error-prone, especially for large image datasets. Here we ask the arguably "circular" question of whether a machine can help us reflect on inappropriate content, answering Question 16 in Datasheets. To this end, we propose to use the information stored in pre-trained transformer models to assist us in the documentation process. Specifically, prompt-tuning based on a dataset of socio-moral values steers CLIP to identify potentially inappropriate content, therefore reducing human labor. We then document the inappropriate images found using word clouds, based on captions generated using a vision-language model. The documentations of two popular, large-scale computer vision datasets-ImageNet and OpenImages-produced this way suggest that machines can indeed help dataset creators to answer Question 16 on inappropriate image content.

show abstract

Section: Inappropriate Image Contentsupporting

confidence: 69%

Section: Issues Arising From Large Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?

Schramowski¹,

Tauchmann²,

Kersting³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, other types of Web content, such as images and memes, require automatic moderation, as they can also be harmful. Only few works have addressed this problem [107,230].…”

Section: Adjacent Researchmentioning

confidence: 99%

Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature

Balayn

Yang

Szlávik

et al. 2021

Trans. Soc. Comput.

View full text Add to dashboard Cite

The automatic detection of conflictual languages (harmful, aggressive, abusive, and offensive languages) is essential to provide a healthy conversation environment on the Web. To design and develop detection systems that are capable of achieving satisfactory performance, a thorough understanding of the nature and properties of the targeted type of conflictual language is of great importance. The scientific communities investigating human psychology and social behavior have studied these languages in details, but their insights have only partially reached the computer science community. In this survey, we aim both at systematically characterizing the conceptual properties of online conflictual languages, and at investigating the extent to which they are reflected in state-of-the-art automatic detection systems. Through an analysis of psychology literature, we provide a reconciled taxonomy that denotes the ensemble of conflictual languages typically studied in computer science. We then characterize the conceptual mismatches that can be observed in the main semantic and contextual properties of these languages and their treatment in computer science works; and systematically uncover resulting technical biases in the design of machine learning classification models and the dataset created for their training. Finally, we discuss diverse research opportunities for the computer science community and reflect on broader technical and structural issues.

show abstract

“…Also, these themes are usually relevant throughout the year. Hence, we address this type of themes by building supervised models [5,8]. The second type of themes is characterized by ill-defined requirements.…”

Section: Non-compliant Product Detectionmentioning

confidence: 99%

SIR: Similar Image Retrieval for Product Search in E-Commerce

Stanley¹,

Vanjara²,

Pan³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We present a similar image retrieval (SIR) platform that is used to quickly discover visually similar products in a catalog of millions. Given the size, diversity, and dynamism of our catalog, product search poses many challenges. It can be addressed by building supervised models to tagging product images with labels representing themes and later retrieving them by labels. This approach suffices for common and perennial themes like "white shirt" or "lifestyle image of TV". It does not work for new themes such as "e-cigarettes", hard-to-define ones such as "image with a promotional badge", or the ones with short relevance span such as "Halloween costumes". SIR is ideal for such cases because it allows us to search by an example, not a pre-defined theme. We describe the steps -embedding computation, encoding, and indexing -that power the approximate nearest neighbor search back-end. We also highlight two applications of SIR. The first one is related to the detection of products with various types of potentially objectionable themes. This application is run with a sense of urgency, hence the typical time frame to train and bootstrap a model is not permitted. Also, these themes are often short-lived based on current trends, hence spending resources to build a lasting model is not justified. The second application is a variant item detection system where SIR helps discover visual variants that are hard to find through text search. We analyze the performance of SIR in the context of these applications.

show abstract

Scalable Detection of Offensive and Non-compliant Content / Logo in Product Images

Cited by 26 publications

References 25 publications

Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?

Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?

Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature

SIR: Similar Image Retrieval for Product Search in E-Commerce

Contact Info

Product

Resources

About