Background: Active surveillance (AS) aims to reduce overtreatment and minimize the negative side effects of radical therapies (i.e., prostatectomy or radiotherapy) while preserving quality of life. However, a substantial proportion of men can experience a decline in sexual function during AS follow-up. The aim of this study was to identify predictors of declining sexual function among men on AS. Methods: Men enrolled from 2008 to 2018 in the South Australian Prostate Cancer Clinical Outcomes Collaborative registry-a prospective clinical registry-were studied. Sexual function outcomes were measured using expanded prostate cancer index composite (EPIC-26) at baseline and 12-months postdiagnosis. Multivariable regression models adjusted for baseline score and other sociodemographic and clinical factors were applied to identify predictors of sexual function score at 12-months. Results: A total of 554 men were included. Variables that showed significant association with decline in sexual function score at 12-months were: having two or more biopsies after diagnosis (mean change score (MCS): -16.3, p < 0.001) compared with no biopsy, higher number of positive biopsy cores (MCS: -1.6, p = 0.004), being in older age category (above 70 vs. below 60: MCS: -16.7, p < 0.001; 65−70 vs. below 60: MCS: -9.7, p = 0.024), having had depression (MCS: -9.0, p = 0.020), and impaired physical function (MCS: -10.0, p = 0.031). Greater socioeconomic advantage (highest vs. lowest quintile: MCS: 15.7, p = 0.022) and year of diagnosis (MCS: 2.6 for every year, p < 0.001) were positively associated with 12-months sexual function score. Neither biopsy type, biopsy timing nor PSA velocity were associated with declines in sexual function. Conclusions: Our findings suggest that multiple factors affected sexual function during AS. Interventions toward reducing the number of biopsies through less invasive monitory approaches, screening for physical and mental well-being, and targeted emotional support and counseling services may be helpful for men on AS.