We get the algorithms of our ground truths: Designing referential databases in digital image processing

Abstract

This article documents the practical efforts of a group of scientists designing an image-processing algorithm for saliency detection. By following the actors of this computer science project, the article shows that the problems often considered to be the starting points of computational models are in fact provisional results of time-consuming, collective and highly material processes that engage habits, desires, skills and values. In the project being studied, problematization processes lead to the constitution of referential databases called ‘ground truths’ that enable both the effective shaping of algorithms and the evaluation of their performances. Working as important common touchstones for research communities in image processing, the ground truths are inherited from prior problematization processes and may be imparted to subsequent ones. The ethnographic results of this study suggest two complementary analytical perspectives on algorithms: (1) an ‘axiomatic’ perspective that understands algorithms as sets of instructions designed to solve given problems computationally in the best possible way, and (2) a ‘problem-oriented’ perspective that understands algorithms as sets of instructions designed to computationally retrieve outputs designed and designated during specific problematization processes. If the axiomatic perspective on algorithms puts the emphasis on the numerical transformations of inputs into outputs, the problem-oriented perspective puts the emphasis on the definition of both inputs and outputs.

Perspectives on algorithmic normativities: engineers, objects, activities

“…The very notions of ''false positive'' or ''false negative'' suppose that an algorithm-based classification can be compared to, and overturned by, a human-based classification. This human-based classification, acting as the reference norm against which all machinebased classifications are to be evaluated, is called the ground truth dataset (Jaton, 2017). The ''dataset'' part here refers to an ensemble of recorded scenes, each containing as many numerical sequences as there are available sensors (i.e.…”

Opinions that matter: the hybridization of opinion and reputation measurement in social media listening software

“…While there is a legitimate preoccupation about the ‘capitalism of surveillance’ (Zuboff, 2018), one research priority should be to understand the genealogy and methodological premises of the device that materially perform said surveillance. Future research should also produce in-depth accounts of the actual uses of big data surveillance technologies, since they cannot be understood simply by studying their general working principles (Jaton, 2017; Seaver, 2017). Historical and ethnographic studies of algorithmic devices are key if we are to understand how data are constituted as a new form of capital by corporate actors.…”

Mass personalization: Predictive marketing algorithms and the reshaping of consumer knowledge

“…To this end, in the continuity of an emerging research stream, I follow as closely as possible the actors involved in the design and circulation of algorithmic models. I thus study predictive marketing as an activity, inscribed in an ecology of practices, rooted within organized work collectives (Christin, 2017;Jaton, 2017). Such an approach goes beyond a strictly semiotic analysis of algorithms and their general spirit, or "fetichization of code" (Chun and Kyong, 2008).…”