Multiple sclerosis (MS) is a neurodegenerative disorder and an autoimmune disease. Until now, observational studies have indicated the association of bone mineral density (BMD) and fracture with the risk of MS. However, these studies indicated inconsistent findings. Until now, genome-wide association studies (GWAS) have been conducted in BMD, fracture, and MS, which provide large-scale datasets to investigate the causal association of BMD and fracture with the risk of MS using the Mendelian randomization (MR) study. Here, we performed an MR study to clarify the causal association between BMD/fracture and the risk of MS using large-scale publicly available GWAS datasets from BMD, fracture, and MS. We first evaluated the bidirectional causal effects of BMD and MS. The main analysis method inverse-variance weighted (IVW) showed no significant causal effect of BMD on the risk of MS (β = 0.058, and p = 1.98E-01), and MS on the risk of BMD (β = −0.001, and p = 7.83E-01). We then evaluated the bidirectional causal effects of fracture and MS. However, we only identified a significant causal effect of fracture on the risk of MS using IVW (β = −0.375, p = 0.002), but no significant causal effect of MS on the risk of the fracture using IVW (β = 0.011, p = 2.39E-01). Therefore, our main analysis method IVW only found a significant causal effect of fracture on MS using the threshold for the statistically significant association p < 0.05/4 = 0.0125. Meanwhile, multivariable MR analyses showed that the causal effect of fracture on MS was independent of smoking, drinking, and obesity, but dependent on BMD. In summary, our MR analysis demonstrates that genetically increased fracture may reduce the risk of MS. Our findings should be further verified and the underlying mechanisms should be further evaluated by future studies.