The Dynamics of Experimentally Induced Criterion Shifts.

Abstract

Investigations of decision making have typically assumed stationarity, even though commonly observed "context effects" are dynamic by definition. Mirror effects are an important class of context effects that can be explained by changes in participants' decision criteria. When easy and difficult conditions are blocked alternately and a mirror effect is observed, participants must repeatedly change their decision criteria. The authors investigated the time course of these criterion changes and observed the buildup of mirror effects on a trial-by-trial basis. The data are consistent with slow, systematic changes in decision criteria that lag behind stimulus changes. The length of this lag is considerable: analysis of a simple dynamic signal-detection model suggests participants take an average of around 14 trials to adjust to new decision environments. This trial-level measurement of experimentally induced changes has implications for traditional blockwise analyses of data and for models of decision making.Keywords: mirror effect, dynamics, context effect, signal detection theory, decision making A common assumption in models of decision making is stationarity. With few exceptions (e.g., Kac, 1966;Rabbit, 1981;Strayer & Kramer, 1994a, 1994bTreisman & Williams, 1984;Vickers & Lee, 1998, models of decision making assume that successive decisions are independent. The assumption of stationarity has proven useful in keeping models simple and tractable and seems reasonable, as most decision-making experiments have used stationary decision-making environments. More recently, there has been a growing focus on nonstationary (dynamic) research. A central feature of most dynamic research in psychology is a focus on behavioral changes triggered by internal events, such as stimulus or response monitoring and error-rate tracking (e.g., Heit, Brockdorff, & Lamberts, 2003;Kelly, Heath, & Longstaff, 2001;Petrov & Anderson, 2005;Rotello & Heit, 2000;Treisman & Williams, 1984; Van Orden, Moreno, & Holden, 2003). Often, these internally induced changes are fast, on the order of seconds (although see also Gilden, Thornton, & Mallon, 1995). The key aspect of internally induced changes is that they can occur at any point during measurement-there is no way to predict their arrival times before the experiment begins.Below, we consider decision environments that are themselves dynamic, experimentally inducing changes in behavior. For example, consider a medical observer making decisions about the nature of tumors (benign vs. malignant) from X-ray photographs. Decision difficulty will change with time, as the patient population or perhaps the picture clarity changes. Observers must dynamically adjust their decision-making processes to reflect changes in the environment: if it becomes easier to identify benign tumors, observers should relax their criterion for identifying malignant tumors. Below, we report an empirical and theoretical investigation of this classic criterion setting problem. We introduce a simple decision model based on signa...

Memory strength and the decision process in recognition memory

“…In our experiments, 80 trials followed the shift in old item strength. In the studies of Brown and Steyvers (2005) and Benjamin and Bawa (2004), far fewer trials were needed to induce criterion shifts. Still, Experiment 4 can shed light on this question, because any effect of strength on initial criterion placement should be observed within the first 80 test trials.…”

“…We attempted to do this while addressing two specific concerns. First, in the context of a lexical decision task, Brown and Steyvers (2005) raised the important point that it takes some time to become aware of changes in test conditions; changes in response bias can lag significantly behind changes in test conditions. In our experiments, 80 trials followed the shift in old item strength.…”

Modeling the adaptation of search termination in human decision making.

“…The ability of the hierarchical confidence model to provide the best predictions for some participants, particularly in direct competition with a reduced model that does not include delays, but is otherwise identical-suggests it is an important phenomenon. Delayed sequential effects in decision making are difficult to study and model (e.g., Gao, Wong-Lin, Holmes, Simen, & Cohen, 2009), and the extent of delay is sometimes captured in dynamic decision making models simply by the inclusion of free parameters (Brown & Steyvers, 2005). The hierarchical model, as with the original SRA model on which it is based, provides a viable alternative.…”

“…As Speekenbrink and Shanks (2010) point out in their empirical and modeling investigation of cue-based categorization, real-world environments change in many ways. Previous work in modeling human decision making has used environments involving gradual drift (e.g., Otto et al, 2010;Rakow & Miller, 2009), patterns consistent with cyclical change (e.g., Yi, Steyvers, & Lee, 2009), step change jumps (e.g., Brown & Steyvers, 2005), and combinations of all of these sort of dynamics (e.g., Speekenbrink & Shanks, 2010). Our experimental design considered only large discrete changes of two types, encouraging a transition from limited to extensive search, and back again.…”

“…There is a large literature on animal and human behavior in dynamic environments, measuring experimentally how changes are detected and environments monitored, and attempting to understand the learning and adaptation processes underlying decision making. (e.g., Biernaskie, Walker, & Gegear, 2009;Brown & Steyvers, 2005;Eliassen, Jørgensen, Mangel, & Giske, 2009;Gallistel, Fairhurst, & Balsam, 2004;Marshall, J, Ashford, Rowcliffe, & G, 2013;Nassar et al, 2012;Otto, Gureckis, Markman, & Love, 2010;Speekenbrink & Shanks, 2010).…”