The ability to switch between goals is the cornerstone of human cognition and behavior. Cognitive control allows for rapid adjustments of cognition in accordance with new goals, but control adjustments come at a cost. This cost is traditionally studied in situations that demand changes to one's task, without necessitating other changes in the control state. Goal flexibility, however, often entails maintaining the same task while adjusting the amount and type of control being allocated to that task. For instance, different stages of a given task might require us to process information more or less efficiently (e.g., by varying levels of attention) and/or respond more or less cautiously (e.g., by varying response thresholds). Across four experiments, we show that such within-task control adjustments incur a performance cost, and that a dynamical systems model can explain the source of these costs. Participants performed a single cognitively demanding task (the color-word Stroop) under varying performance goals (e.g., to be fast or to be accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model predicts and our empirical findings confirm that people will under-shoot their target control state more (i.e., exhibit larger adjustment costs) when (a) goals switch rather than remain fixed over a block (Study 1); (b) target control states are more distant from one another (Study 2); (c) less time is given to adjust to the new goal (Study 3); and (d) when anticipating having to switch goals more frequently (Study 4). Our findings demonstrate that there is a cost to adjusting control to meet one's goal - even in the absence of a task change - and show that this cost can emerge directly from the dynamics of control adjustment. In so doing, they shed new light on the sources of and constraints on flexibility in human goal-directed behavior.