The burden of depression and anxiety disorders is greater in women, and female sex hormones have been shown to affect mood. Psychological side effects of hormonal contraception (HC) are also a common complaint in the clinic, but few previous studies have investigated this subject. We therefore wanted to investigate whether use of HC was associated with adverse psychological health outcomes, and whether this association was modified by age. All women aged 12–30 years on 31 December 2010, residing in Sweden for at least four years and with no previous psychiatric morbidity (n = 815 662), were included. We followed the women from their first HC use (or 31 December 2010, if they were non-users) at baseline, until a prescription fill of psychotropic drugs or the end of the one-year follow-up. We performed age-stratified logistic regression models and estimated odds ratios (OR) to measure the association between different HC methods and psychotropic drug use, as well as the area under the receiver operating curve to estimate discriminatory accuracy of HC in relation to psychotropic drugs. Overall, we found an association between HC and psychotropic drugs (adjusted OR 1.34, 95% confidence interval [CI] 1.30–1.37). In the age-stratified analysis, the strongest association was found in adolescent girls (adjusted OR 3.46, 95% CI 3.04–4.94 for age 12 to 14 years), while it was non-existent for adult women. We conclude that hormonal contraception is associated with psychotropic drug use among adolescent girls, suggesting an adverse effect of HC on psychological health in this population.