Background: Risky sexual behaviors remain common among adolescents regardless of those with comprehensive knowledge of safer sex practices. Self-efficacy has been shown to have a positive relationship with safer sex practices. Thus, investigating self-efficacy, and enhancing it to agency is important. The current study explores the predictors of self-efficacy for avoiding risky sexual behaviors and what limits agency among sexually active adolescents (15–19 years) in Northern Uganda. Methods: The study consisted of a sub-sample of 396 sexually active adolescents (145 in school, 251 out of school) interviewed as part of a household survey for the program on Advancing Sexual Reproductive Health and Rights in Northern Uganda. Poisson and Poisson regression models with survey weights were implemented in Stata. Results: A total of 94% of male and 64% of female adolescents reported self-efficacy to avoid unsafe sex, including using condoms and avoiding multiple sexual partnerships or transactional sex. At multivariable analysis, a higher proportion of adolescents who listened to a radio or television program about sexual and reproductive health within the past 12 months had self-efficacy as compared to others (PR = 1.13, p-value = 0.002). Similarly, higher proportions of adolescents who knew all the sexual and reproductive health (SRH) rights (PR = 1.33, p-value = 0.007) and who had comprehensive knowledge about pregnancy, prevention of sexual transmission infections, and sources of SRH services (PR = 1.24, p-value = 0.013) had self-efficacy as compared to others. However, among those who reported self-efficacy, 42% of the girls and 53% of the boys could not uphold their self-efficacy in actual sexual encounters in the past 12 months. Partner’s refusal or girls’ fear to ask their sexual partner to use a condom were commonly cited reasons. Alcohol consumption was associated with failure to act on one’s self-efficacy (RR = 0.74, p-value = 0.048). Conclusions: Programs should target self-efficacy beliefs and attempt to enhance them into agency by increasing positive and decreasing negative expectations associated with risky sexual behavior.