Contextualist model evaluation: models in financial economics and index funds

Abstract

Philosophers of science typically focus on the epistemic performance of scientific models when evaluating them. Analysing the effects that models may have on the world has typically been the purview of sociologists of science. We argue that the reactive (or “performative”) effects of models should also figure in model evaluations by philosophers of science. We provide a detailed analysis of how models in financial economics created the impetus for the growing importance of the phenomenon of “passive investing” in financial markets. Considering this case motivates the position that we call contextualism about model evaluation, or model contextualism for short. Model contextualism encompasses standard analyses of the epistemic performance of the model, but also includes their reactive aspects. It entails identifying the epistemic and contextual import of the model, the ways in which a model can engender change in the world (which we call the channels of transmission), and the interactions between the epistemic and reactive import of a model.

Managing Performative Models

“…3 This is different from other cases discussed in the literature where models co-shape a target through agents using the models to gain new forms of epistemic and/or practical access to or control over a target (see Guala 2007;Vergara-Fernández, Heilmann, and Szymanowska 2023). even though X would have been X ¼ x if the model had predicted differently, or had not been used to predict X (or if the causal pathways between model and target had been disrupted) (MacKenzie 2006).…”

“…Second, a performative effect is the difference to a target system that is, causally, due to the model and its outputs. Third, in understanding model performativity, it is important to note that models are rarely performative as such but typically become performative only when embedded in a concrete context of use, which establishes causal connections between the model and its target (see Vergara-Fernández, Heilmann, and Szymanowska 2023). It often takes a user who does something with a model (e.g., derive and publicize a prediction) to establish such a connection.…”

“…Model performativity is now situated in a rich and growing conceptual forest with cognate notions tracking several related phenomena, including reactivity, reflexivity, interactivity, and others (see Buck 1963; Godman and Marchionni 2022; Henshel 1993; Jiménez-Buedo 2021; Vergara-Fernández, Heilmann, and Szymanowska 2023). I will not explore these related concepts and the interesting arguments they ground, nor offer a general account of performativity or trace its rich intellectual history (see Callon and Roth 2021; Guala 2007; Mäki 2013; MacKenzie 2006; Perdomo et al 2021; van Basshuysen 2022).…”

“…Scientific models can be performative: in addition to serving various epistemic purposes, they can also causally affect phenomena, such as when agents’ behaviors change in response to model predictions. In recent years, philosophers have made substantial progress in delineating different forms of performativity and characterizing the problems they can pose, such as when the forecasts researchers derive from models are self-defeating and compromise models’ epistemic functioning (Avery et al 2020b; Godman and Marchionni 2022; Jiménez-Buedo 2021; Northcott 2022; Tee 2019; van Basshuysen 2022; van Basshuysen et al 2021; Vergara-Fernández, Heilmann, and Szymanowska 2023; Winsberg and Harvard 2022).…”

“…There, we argued that performativity can sometimes be understood as a good-making model attribute, e.g., when predictions derived from models, such as that critical care demand will exceed capacity, steer the public’s behavior in desirable directions. Our appraisal approach hence understands models as tools that have both epistemic and performative capabilities, both of which should be considered in model evaluation, and permits (some forms of) performativity to count as a good-making feature of models (see Vergara-Fernández, Heilmann, and Szymanowska 2023 for related proposals).…”