(1) Background: This study aimed to investigate the relationship of triglyceride glucose–body mass index (TyG-BMI) with bone mineral density (BMD), femoral neck geometry, and risk of fracture in middle-aged and elderly Chinese individuals. (2) Methods: A total of 832 nondiabetic individuals were selected from the prospective population-based HOPE cohort. All individuals underwent DXA for assessment of BMD at the lumbar spine, femoral neck, and total hip, as well as femoral neck geometry. The 10-year probabilities of both major osteoporotic (MOFs) and hip fractures (HFs) were calculated. (3) Results: Cortical thickness, compression strength index, cross-sectional moment of inertia, cross-sectional area, section modulus, and 25(OH)D levels were significantly lower in women (all p < 0.001). The presence of osteoporosis was related to age, BMI, BMD and femoral neck geometry, TyG-BMI, MOF, and HF. TyG-BMI was positively correlated with BMD. In men, TyG-BMI showed significant negative correlation with HF but not with MOF, the correlation exists only after adjusting for other variables in women. Femoral neck geometries were significantly impaired in individuals with low TyG-BMI. (4) Conclusion: TyG-BMI is positively associated with BMD and geometry, and negatively associated with risk of fracture in nondiabetic middle-aged and elderly Chinese men and women.