Reduced sleep duration and sleep deprivation have been associated with cognitive impairment as well as decreased white matter integrity as reported by experimental studies. However, it is largely unknown whether differences in sleep duration and sleep quality might affect microstructural white matter and cognition. Therefore, the present study aims to examine the cross‐sectional relationship between sleep duration, sleep quality, and cognitive performance in a naturalistic study design, by focusing on the association with white matter integrity in a large sample of healthy, young adults. To address this, 1,065 participants, taken from the publicly available sample of the Human Connectome Project, underwent diffusion tensor imaging. Moreover, broad cognitive performance measures (NIH Cognition Toolbox) and sleep duration and quality (Pittsburgh Sleep Quality Index) were assessed. The results revealed a significant positive association between sleep duration and overall cognitive performance. Shorter sleep duration significantly correlated with fractional anisotropy (FA) reductions in the left superior longitudinal fasciculus (SLF). In turn, FA in this tract was related to measures of cognitive performance and was shown to significantly mediate the association of sleep duration and cognition. For cognition only, associations shift to a negative association of sleep duration and cognition for participants sleeping more than 8 hr a day. Investigations into subjective sleep quality showed no such associations. The present study showed that real‐world differences in sleep duration, but not subjective sleep quality are related to cognitive performance measures and white matter integrity in the SLF in healthy, young adults.