Pupil Diameter Tracks the Exploration–Exploitation Trade-off during Analogical Reasoning and Explains Individual Differences in Fluid Intelligence

Abstract

Abstract■ The ability to adaptively shift between exploration and exploitation control states is critical for optimizing behavioral performance. Converging evidence from primate electrophysiology and computational neural modeling has suggested that this ability may be mediated by the broad norepinephrine projections emanating from the locus coeruleus (LC) [Aston-Jones, G., & Cohen, J. D. An integrative theory of locus coeruleus-norepinephrine function: Adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403-450, 2005]. There is also evidence that pupil diameter covaries systematically with LC activity. Although imperfect and indirect, this link makes pupillometry a useful tool for studying the locus coeruleus norepinephrine system in humans and in high-level tasks. Here, we present a novel paradigm that examines how the pupillary response during exploration and exploitation covaries with individual differences in fluid intelligence during analogical reasoning on Raven's Advanced Progressive Matrices. Pupillometry was used as a noninvasive proxy for LC activity, and concurrent think-aloud verbal protocols were used to identify exploratory and exploitative solution periods. This novel combination of pupillometry and verbal protocols from 40 participants revealed a decrease in pupil diameter during exploitation and an increase during exploration. The temporal dynamics of the pupillary response was characterized by a steep increase during the transition to exploratory periods, sustained dilation for many seconds afterward, and followed by gradual return to baseline. Moreover, the individual differences in the relative magnitude of pupillary dilation accounted for 16% of the variance in Advanced Progressive Matrices scores. Assuming that pupil diameter is a valid index of LC activity, these results establish promising preliminary connections between the literature on locus coeruleus norepinephrine-mediated cognitive control and the literature on analogical reasoning and fluid intelligence. ■

Explore or reset? Pupil diameter transiently increases in self-chosen switches between cognitive labor and leisure in either direction

“…These changes in pupil baseline levels and pupil dilations were short-lived. Our findings extend previous studies that found similar pupil changes when people disengaged from (and restarted) an auditory discrimination task (Gilzenrat et al, 2010), when people shifted between multiple bandit gambling machines (Jepma & Nieuwenhuis, 2011) and when people shifted between strategies in solving Raven's Matrices (Hayes & Petrov, 2016). By contrast to these previous studies, in our research, participants chose between two motivationally different tasks-namely an effortful, but profitable 2-back (labor) and an effortless, but unprofitable attractiveness rating task (leisure).…”

“…In sum, building on our findings and on recent insights in NE function, it is possible that previous experiments about shifts from exploitation to exploration (Gilzenrat et al, 2010;Hayes & Petrov, 2016;Jepma & Nieuwenhuis, 2011;Kane et al, 2017;Pajkossy et al, 2017) observed NE facilitating the execution of task switching, rather than NE facilitating motivational shifts. By contrast to these previous studies, our paradigm used two qualitatively distinct tasks, with distinct motivational properties (i.e., labor vs. leisure).…”

“…At any time, participants could reset task settings by pressing an "escape" button, which the authors interpreted as exploration behavior. Changes in pupil dilation were also observed in transitions between bandit gambling machines (Jepma & Nieuwenhuis, 2011) and in a task that required people to solve Raven's Matrices (Hayes & Petrov, 2016). Overall, in these studies, exploitation was characterized by relatively low baseline pupil diameter, combined with large pupil dilations in response to task stimuli.…”

A learning perspective on individual differences in skilled reading: Exploring and exploiting orthographic and semantic discrimination cues.

“…Across different tasks (Stafford & Dewar, 2015;Hayes & Petrov, 2015;Gureckis & Love, 2009), humans show variable behavioural strategies, with initial exploration of possible responses and later exploitation of efficient ones. From computer gaming to analogical reasoning and decision making, it has been shown that participants who initially show more diversity across trials (and possibly commit more errors) show better overall performance.…”

“…First, van Ravenzwaaij, Brown, and Wagenmakers (2011) showed that the speed of information processing (i.e., Òdrift rateÓ from the Diffusion Model: Ratcliff, 1978;and Ratcliff, Schmiedek, & McKoon, 2008), correlates well with the reaction timeÕs standard deviation and somewhat less consistently with its mean (see also Baumeister, 1998). Second, Stafford and Dewar (2014) reported that game players who exhibit greater initial variability in performance achieved a higher overall score, and explained this pattern in terms of the exploration/exploitation trade-off (for similar results in other domains see Stafford et al, 2012;Gureckis & Love, 2009;Hayes & Petrov, 2015). By combining the findings of Stafford and Dewar (2014) and van Ravenzwaaij, Brown, and Wagenmakers (2011) with our two measures of individual differences in speed we are in a position to hypothesize that (a) an individualÕs processing speed will be indicated by their deviations, both in average speed and in the change in speed; and that (b) a positive change in speed (i.e., acceleration) may disguise more variable performance at the beginning of the experiment, in order to explore and then exploit rewarding behaviour at a later point in time.…”