In resource-poor countries where access to infertility care is limited, women may turn to traditional medicine to achieve motherhood. It is unknown whether Sierra Leonean women with such condition use herbal medicine. This study investigates the prevalence and factors associated with herbal medicine use among women seeking care for infertility. This was a questionnaire-based cross-sectional study conducted among women seeking care for infertility at various clinics within Freetown, Sierra Leone. Data analysis included Chi-square tests and logistic regression. Out of the 167 women that participated, 36.5% used herbal medicine for infertility treatment. Women with no formal (AOR 4.03, CL: 1.38–11.76, p = 0.011), primary education (AOR: 6.23, CL: 2.02–19.23, p = 0.001) and those that visited a traditional medicine practitioner (AOR: 20.05, CL: 2.10–192.28, p = 0.009) as well as women suffering from other reproductive health problems (AOR: 2.57, CL: 1.13–5.83, p = 0.024) were more likely to use herbal medicines. Friends and family (n = 57, 96.7%) were the main influencers of herbal medicine use. Only (n = 12) 19.7% of users disclosed their status to their healthcare provider. Over half (n = 32, 52.5%) could not remember the name of the herb they used. Luffa acutangula (n = 29, 100%) was the herbal medicinal plant users could recall. Herbal medicine use among women seeking care for infertility in Freetown is common. Healthcare providers should be aware of the potential dyadic use of herbal and allopathic medicines by their patients and be knowledgeable about commonly used herbal remedies as well as being proactive in communicating the potential risks and benefits associated with their use.