The quadrupole moment of a non-fullerene acceptor (NFA) generated by the constituent electron donor (D) and acceptor (A) units is a significant factor that affects the charge separation (CS) and charge recombination (CR) processes in organic photovoltaics (OPVs). However, its impact on p-type polymer domains remains unclear. In this study, we synthesized p-type molecules, namely acceptor–donor–acceptor (ADA) and donor–acceptor–donor (DAD), which are components of the benchmark PM6 polymer (D: benzodithiophene and A: dioxobenzodithiophene). Planar heterojunction films, a model of bulk heterojunction, were prepared using ADA, DAD, and PM6 as the bottom p-type layers and Y6 NFA as the top n-type layer. Flash-photolysis time-resolved microwave conductivity, femtosecond transient absorption spectroscopy, and quantum mechanical calculations were employed to probe the charge carrier dynamics. Our findings reveal that while the subtle difference in quadrupole moment and energy gradient of the p-type materials has a minimal influence on CS, the molecular type (ADA or DAD) significantly affects the bulk CR. This study expands the understanding of how the p-type component and its conformation at the p/n interface impact the CS and CR in OPVs, highlighting the critical role of molecular donors in optimizing device performance.