Background Body composition alterations may participate in the pathophysiological processes of type 2 diabetes (T2D). A sedentary lifestyle may be responsible for alterations of body composition and adverse consequences, but on which body composition of patients with T2D and to what extent the sedentary lifestyle has an effect have been poorly investigated. Methods We recruited 402 patients with T2D for this cross-sectional study. All patients received questionnaires to evaluate sedentary time and were further divided into three subgroups: low sedentary time (LST, < 4 h, n = 109), middle sedentary time (MST, 4–8 h, n = 129) and high sedentary time (HST, > 8 h, n = 164). Each patient underwent a dual energy X-ray absorptiometry (DXA) scan to detect body composition, which included body fat percentage (B-FAT), trunk fat percentage (T-FAT), appendicular skeletal muscle index (ASMI), lumbar spine bone mineral density (BMD) (LS-BMD), femoral neck BMD (FN-BMD), hip BMD (H-BMD) and total BMD (T-BMD). Other relevant clinical data were also collected. Results With increasing sedentary time (from the LST to HST group), B-FAT and T-FAT were notably increased, while ASMI, LS-BMD, FN-BMD, H-BMD and T-BMD were decreased (p for trend < 0.01). After adjustment for other relevant clinical factors and with the LST group as the reference, the adjusted mean changes [B (95% CI)] in B-FAT, T-FAT, ASMI, LS-BMD, FN-BMD, H-BMD and T-BMD in the HST group were 2.011(1.014 to 3.008)%, 1.951(0.705 to 3.197)%, − 0.377(− 0.531 to − 0.223) kg/m2, − 0.083(− 0.124 to − 0.042) g/cm2, − 0.051(− 0.079 to − 0.024) g/cm2, − 0.059(− 0.087 to − 0.031) g/cm2 and − 0.060(− 0.088 to − 0.033) g/cm2, p < 0.01, respectively. Conclusions A sedentary lifestyle may independently account for increases in trunk and body fat percentage and decreases in appendicular skeletal muscle mass and BMD of the lumbar spine, femoral neck, hip and total body in patients with T2D.