The association between hypogonadism, quality of life (QoL), and erectile dysfunction (ED) among the middle-aged and aged male in Taiwan is evaluated. A total of 680 study subjects aged X40 years old were recruited from Northern (n ¼ 276), Middle (n ¼ 238), and Southern (n ¼ 202) Taiwan, respectively. ED was diagnosed by score of International Index of Erectile Function (IIEF-5). Taiwan version questionnaire for QoL includes domain 1 (physical domain), domain 2 (psychological domain), domain 3 (social relationship domain), and domain 4 (environmental domain) was used to measure QoL. Blood hormones, including FSH, LH, Prolactin, SHBG, total testosterone (TT), calculated free testosterone (cFT), and bioavailable testosterone (Bio-T), were determined. Logistic regression analysis was used to estimate crude and multivariate-adjusted odds ratio of risk factors and its 95% confidence interval. A significantly inverse association between concentration of serum cFT and Bio-T, and severity of ED was observed. Scores of QoL of Domain 1-4 were significantly decreased with the increament of severity of ED. Significant correlations were found between IIEF scores and four domains of QoL, respectively. After adjustment for age, cFT and Bio-T, study subjects with ED (IIEFp21) would have significantly high risk of low level of QoL in four domains. In conclusion, a significant association between low levels of serum calculated cFT, Bio-T, and severity of ED was found. In addition, abnormal erectile function significantly associated with low level of QoL in four domains.