This study used data collected from an online survey study on coronavirus disease 2019 (COVID-19) in Taiwan to examine changes in sex life during the pandemic and the factors affecting such changes. In total, 1954 respondents were recruited from a Facebook advertisement. The survey inquired changes in sex life during the pandemic, including satisfaction with the individual’s sex life, frequency of sexual activity, frequency of sex-seeking activity, and frequency of using protection for sex. The associations of change in sex life with risk perception of COVID-19, general anxiety, gender, age, and sexual orientation were also examined. For each aspect of their sex life, 1.4%–13.5% of respondents reported a decrease in frequency or satisfaction, and 1.6%–2.9% reported an increase in frequency or satisfaction. Risk perception of COVID-19 was significantly and negatively associated with frequencies of sexual and sex-seeking activities. Higher general anxiety was significantly and negatively associated with satisfaction of sex life and frequencies of sexual and sex-seeking activities. Sexual minority respondents were more likely to report decreased satisfaction with sex life and frequencies of sexual activity and sex-seeking activities during COVID-19. Health care providers should consider these factors when developing strategies for sexual wellness amid respiratory infection epidemics.