Mind wandering is a universal phenomenon in which our attention shifts away from the task at hand toward task-unrelated thoughts. Despite it inherently involving a shift in mental set, little is known about the role of cognitive flexibility in mind wandering. In this article we consider the potential of cognitive flexibility as a mechanism for mediating and/or regulating the occurrence of mind wandering. Our review begins with a brief introduction to the prominent theories of mind wandering—the executive failure hypothesis, the decoupling hypothesis, the process-occurrence framework, and the resource-control account of sustained attention. Then, after discussing their respective merits and weaknesses, we put forward a new perspective of mind wandering focused on cognitive flexibility, which provides an account more in line with the data to date, including why older populations experience a reduction in mind wandering. After summarizing initial evidence prompting this new perspective, drawn from several mind-wandering and task-switching studies, we recommend avenues for future research aimed at further understanding the importance of cognitive flexibility in mind wandering.