Aims/hypothesis The study aimed to quantitatively summarise the dose–response relationships between cardiorespiratory fitness and muscular strength on the one hand and risk of type 2 diabetes on the other and estimate the hypothetical benefits associated with population-wide changes in the distribution of fitness. Methods We performed a systematic review with meta-analysis. The PubMed and EMBASE electronic databases were searched from inception dates to 12 December 2018 for cohort studies examining the association of cardiorespiratory fitness or muscular strength with risk of incident type 2 diabetes in adults. The quality of included studies was evaluated using the Newcastle–Ottawa Scale. Results Twenty-two studies of cardiorespiratory fitness and 13 studies of muscular strength were included in the systematic review with both exposures having ten estimates available for the primary adiposity- or body size-controlled meta-analysis. In random-effects meta-analysis including 40,286 incident cases of type 2 diabetes in 1,601,490 participants, each 1 metabolic equivalent (MET) higher cardiorespiratory fitness was associated with an 8% (95% CI 6%, 10%) lower RR of type 2 diabetes. The association was linear throughout the examined spectrum of cardiorespiratory fitness. In 39,233 cases and 1,713,468 participants each 1 SD higher muscular strength was associated with a 13% (95% CI 6%, 19%) lower RR of type 2 diabetes. We estimated that 4% to 21% of new annual cases of type 2 diabetes among 45–64-year-olds could be prevented by feasible and plausible population cardiorespiratory fitness changes. Conclusions/interpretation Relatively small increments in cardiorespiratory fitness and muscle strength were associated with clinically meaningful reductions in type 2 diabetes risk with indication of a linear dose–response relationship for cardiorespiratory fitness. Registration: PROSPERO (CRD42017064526). Electronic supplementary material The online version of this article (10.1007/s00125-019-4867-4) contains peer-reviewed but unedited supplementary material, which is available to authorised users.