Objectives This study proposes a regression model for the phantomless Hounsfield units (HU) to bone mineral density (BMD) conversion including patient physical factors and analyzes the accuracy of the estimated BMD values. Methods The HU values, BMDs, circumferences of the body, and cross-sectional areas of bone were measured from 39 quantitative computed tomography images of L2 vertebrae and hips. Then, the phantomless HU-to-BMD conversion was derived using a multiple linear regression model. For the statistical analysis, the correlation between the estimated BMD values and the reference BMD values was evaluated using Pearson's correlation test. Voxelwise BMD and finite element analysis (FEA) results were analyzed in terms of root-mean-square error (RMSE) and strain energy density, respectively. Results The HU values and circumferences were statistically significant (p < 0.05) for the lumbar spine, whereas only the HU values were statistically significant (p < 0.05) for the proximal femur. The BMD values estimated using the proposed HU-to-BMD conversion were significantly correlated with those measured using the reference phantom: Pearson's correlation coefficients of 0.998 and 0.984 for the lumbar spine and proximal femur, respectively. The RMSEs of the estimated BMD values for the lumbar spine and hip were 4.26 ± 0.60 (mg/cc) and 8.35 ± 0.57 (mg/cc), respectively. The errors of total strain energy were 1.06% and 0.91%, respectively. Conclusions The proposed phantomless HU-to-BMD conversion demonstrates the potential of precisely estimating BMD values from CT images without the reference phantom and being utilized as a viable tool for FEA-based quantitative assessment using routine CT images.