A spate of recent work demonstrates that humans seek to avoid the expenditure of cognitive effort, much like physical effort or economic resources. Less is clear, however, about the circumstances dictating how and when people decide to expend cognitive effort. Here we adopt a popular theory of opportunity costs and response vigor and to elucidate this question. This account, grounded in Reinforcement Learning, formalizes a trade-off between two costs: the harder work assumed necessary to emit faster actions and the opportunity cost inherent in acting more slowly (i.e., the delay that results to the next reward and subsequent rewards). Recent work reveals that the opportunity cost of time-operationalized as the average reward rate per unit time, theorized to be signaled by tonic dopamine levels, modulates the speed with which a person responds in a simple discrimination tasks. We extend this framework to cognitive effort in a diverse range of cognitive tasks, for which 1) the amount of cognitive effort demanded from the task varies from trial to trial and 2) the expenditure of cognitive effort holds measureable consequences in terms of accuracy and response time. In the domains of cognitive control, perceptual decision-making, and task-switching, we found that subjects tuned their level of effort exertion in accordance with the experienced average reward rate: when the opportunity cost of time was high, subjects made more errors and responded more quickly. That is, expenditure of cognitive effort appeared to be modulated by the opportunity cost of time. Further, and consistent with our account, the strength of this modulation was predicted by individual differences in efficacy of cognitive control. Taken together, our results elucidate the circumstances dictating how and when people expend cognitive effort.