(1) Background: Nearly one out of ten Ghanaian female adolescents aged 15–19 has experienced childbearing in urban settlements compared to twice this number in the rural populations due to unintended pregnancies. This study assessed the linkages between knowledge, attitudes, and use of contraceptives and adolescent pregnancy in one of the highly affected Municipalities (i.e., Komenda-Edina-Eguafo Abrem [KEEA]) in Ghana. (2) Methods: Employing a facility-based sampling method, 378 female adolescents aged 15–19 were selected. Unadjusted odds ratio (uOR) and adjusted odds ratio (aOR) at 95% confidence intervals (CI) and p-values were used for significant variables at p < 0.05. (3) Results: Pregnant adolescents were 2 times more likely to indicate that the procedure of procuring contraceptives is quite uncomfortable (aOR = 2.42, 95% CI = [1.29–4.55]; p = 0.006). Also, pregnant adolescents were 5 times more likely to have ever used traditional contraceptive methods than their non-pregnant counterparts (aOR = 5.02, 95% CI = [2.60–9.71]; p < 0.001). On the contrary, pregnant adolescents had lower odds of indicating that contraceptives are for only married people (aOR = 0.38, 95% CI = [0.20–0.70]; p = 0.002) and that it feels bad to receive contraceptive information from parents and relatives than non-pregnant adolescents (aOR = 0.42, 95% CI = [0.24–0.74]; p = 0.003). Pregnant adolescents were less likely to use modern contraceptives than their non-pregnant adolescents (aOR = 0.18, 95% CI = [0.11–0.31]; p < 0.001). (4) Conclusions: The findings indicate that female adolescents’ use of traditional contraceptives is associated with the risk of pregnancy in KEEA Municipality within the Central Region of Ghana. However, adolescents who had the perception that contraceptives are for married people and those who used modern contraceptives were less likely to get pregnant. Government and non-governmental organizations in Ghana should implement educational policies and programmes aimed at educating sexually-active female adolescents on modern contraceptives and the need to use them to prevent pregnancies. The basis for such policies and programmes should be based on evidence that compared to traditional contraceptives, modern contraceptives are more effective. In addition, there is the need to provide accurate information regarding the use of contraceptives to adolescents that will help change their attitudes towards the use of contraceptives.