Improvisation is ubiquitous in life. It deserves, we suggest, to occupy a more central role in cognitive science. In the current paper, we take the case of jazz improvisation as a rich model domain from which to explore the nature of improvisation and expertise more generally. We explore the activity of the jazz improviser against the theoretical backdrop of Dreyfus's account of expertise as well as of enactivist and 4E accounts of cognition and action. We argue that enactivist and 4E accounts provide a rich source of insights on improvisation that go beyond Dreyfus's notion of skilled coping, for example, through the central enactivist notion of "sense-making". At the same time, however, we see improvisation also as suggesting an extension of enactivist theory. We see expert improvisers, in music and in life, as walking on a path of open-ended expansion of their mindful experiential relation with their doing. At the heart of an improviser's expertise (and of day-to-day living), we propose, lies a form of "higher-level inner sense-making" that spontaneously creates novel forms of agentive goal-directedness in the moment. Our account thus supplants Dreyfus's idea of the ego-less absorbed expert by that of a mindful (i.e. present in the moment) improviser enacting spontaneous expressions of herself, in music or in life.