Polycystic ovary syndrome (PCOS) is one of the most common endocrine diseases in women of reproductive age and features complex pathological symptoms and mechanisms. Existing medical treatments have, to some extent, alleviated the deterioration of PCOS. However, these strategies only temporarily control symptoms, with a few side effects and no preventive effect. Phytochemicals extracted from medicinal herbs and plants are vital for discovering novel drugs. In recent years, many kinds of research have proven that phytochemicals isolated from traditional Chinese medicine (TCM) and medicinal plants show significant potential in preventing, alleviating, and treating PCOS. Nevertheless, compared to the abundance of experimental literature and minimal specific-topic reviews related to PCOS, there is a lack of systematic reviews to summarize these advancements in this promising field. Under this background, we systematically document the progress of bioactive phytochemicals from TCM and medicinal plants in treating PCOS, including flavonoids, polyphenols, and alkaloids. According to the literature, these valuable phytochemicals demonstrated therapeutic effects on PCOS supported by in vivo and in vitro experiments, mainly depending on anti-inflammatory, antioxidation, improvement of hormone disorder and insulin resistance (IR), and alleviation of hyperinsulinemia. Based on the current progress, future research directions should emphasize 1) exploring bioactive phytochemicals that potentially mediate bone metabolism for the treatment of PCOS; 2) improving unsatisfactory bioavailability by using advanced drug delivery systems such as nanoparticles and antibody-conjugated drugs, as well as a chemical modification; 3) conducting in-depth research on the pathogenesis of PCOS to potentially impact the gut microbiota and its metabolites in the evolution of PCOS; 4) revealing the pharmacological effects of these bioactive phytochemicals on PCOS at the genetic level; and 5) exploring the hypothetical and unprecedented functions in regulating PCOS by serving as proteolysis-targeting chimeras and molecular glues compared with traditional small molecule drugs. In brief, this review aims to provide detailed mechanisms of these bioactive phytochemicals and hopefully practical and reliable insight into clinical applications concerning PCOS.