Training Computers to See Internet Pornography: Gender and Sexual Discrimination in Computer Vision Science

Abstract

This article critically examines computer vision–based pornography filtering (CVPF), a subfield in computer science seeking to train computers on how to recognize the difference between digital pornographic images and nonpornographic images. Based on a review of 102 peer-reviewed CVPF articles, we argue that CVPF has as a whole trained computers to “see” a very specific, idealized form of pornography: pictures of lone, thin, naked women. The article supports this argument by closely reading the algorithms proposed in the CVPF literature and quantitatively analyzing the images included as illustrations of these algorithms. Drawing on pornography studies, we also compare the CVPF pornographic imagination with “noisy” pornography that exceeds computer vision. Ultimately, the article argues that this very narrow imagination of porn in CVPF reflects and reinforces larger gender and sexual inequalities in the technology industry as a whole.

Data and its (dis)contents: A survey of dataset development and use in machine learning research

“…For example, a survey of literature on computer vision systems for detecting pornography found that the task is largely framed around detecting the features of thin, nude, female-presenting bodies with little body hair, largely to the exclusion of other kinds of bodiesthereby implicitly assuming a relatively narrow and conservative view of pornography that happens to align with a straight male gaze. 25 In an examination of the person categories within the Image-Net dataset, 3 Crawford and Paglen 26 uncovered millions of images of people that had been labeled with offensive categories, including racial slurs and derogatory phrases. In a similar vein, Birhane and Prabhu 27 examined a broader swath of image classification datasets that were constructed using the same cate-gorical schema as ImageNet, finding a range of harmful and problematic representations, including non-consensual and pornographic imagery of women.…”

Content moderation, AI, and the question of scale

“…But this data is an artifact of the policies and judgments of the platform's existing moderation arrangements. An effective tool may learn to make the same kinds of distinctions as before (Binns et al, 2017;Gehl et al, 2017). But while consistency might sound like a good thing, these policies should actually adapt over time (Sinnreich, 2018).…”

Puritanisme sexuel et capitalisme numérique

“…Les seins sont ainsi plus facilement détectés en tant que « contenus sexuels offensants » que les pénis. Cela nous renvoie au problème plus large du biais hétérosexuel masculin blanc dans la conception des logiciels informatiques [Gehl et al, 2017].…”