Korean sexual minority women (SMW) often experience discrimination, but their health-related quality of life (HRQoL) remains to be investigated. Therefore, we aimed to assess the levels of mental and physical HRQoL of Korean SMW and their influencing factors using data from the Korean Sexual Minority Women’s Health Study (2017) in a cross-sectional study, which included lesbian and bisexual females (N = 736; age ≥19 years). The HRQoL was measured using SF-36v2®; moreover, separate multiple linear regression analyses were conducted to identify the factors influencing mental and physical HRQoL. The physical and mental HRQoL scores were average (52.38 ± 7.65) and low (38.33 ± 12.64), respectively. Significant factors influencing the physical HRQoL were bisexuality, minority stress, perceived social support, and physical activity. The same factors—apart from physical activity—were associated with mental HRQoL. Therefore, to improve the HRQoL of SMW, it is necessary to lower their minority stress and increase social support. Moreover, special attention is needed regarding bisexual women in Korea.