There is still considerable controversy surrounding the impact of mastication on obesity. The aim of this study was to identify the interplay between the masticatory muscles, teeth, and general muscular fitness and how they contribute to body adiposity in a general German population. This cross-sectional study included 616 participants (300 male, 316 female, age 31–93 years) from the population-based Study of Health in Pomerania. The cross-sectional areas of the masseter, medial and lateral pterygoid muscles were measured using magnetic resonance imaging (MRI), muscular fitness assessed by hand grip strength (HGS) and body fat distribution was measured by bioelectrical impedance analysis (BIA) and MRI. The overall prevalence of obesity was high in our cohort. The cross-sectional area of the masseter muscles was positively associated with the number of teeth, body mass index (BMI) and HGS, and negatively associated with the BIA-assessed body fat when adjusted for age, sex, teeth, and BMI. Especially the correlation was strong (p < 0.001). Analogous relationships were observed between the masseter, HGS and MRI-assessed subcutaneous fat. These associations were most pronounced with masseter, but also significant with both pterygoid muscles. Though the masticatory muscles were affected by the number of teeth, teeth had no impact on the relations between masseter muscle and adiposity. Physical fitness and masticatory performance are associated with body shape, controlled and directed by the relevant muscles.