From the industrial perspective, poly(3-hexylthiophene) (P3HT) is one of the most attractive donor materials in organic photovoltaics. The large bandgap in P3HT makes it particularly promising for efficient indoor light harvesting, a unique advantage of organic photovoltaic (PV) devices, and this has started to gain considerable attention in the field of PV technology. In addition, the up-scalability and long material stability associated with the simple chemical structure make P3HT one of the most promising materials for the mass production of organic solar cells. However, the solar cells based on P3HT has a low power conversion efficiency (PCE), which is less than 11%, mainly due to significant voltage losses. In this study, we identified the origin of the high quantum efficiency and voltage losses in the P3HT:non-fullerene based solar cells, and we proposed a strategy to reduce the losses. More specifically, we observed that: 1) the non-radiative decay rate of the charge transfer (CT) states formed at the donor-acceptor interfaces was much higher for the P3HT:nonfullerene solar cells than that for the P3HT:fullerene solar cells, which was the main reason for the more severely limited photovoltage; 2) the origin of the high non-radiative decay rate in the P3HT:non-fullerene solar cell could be ascribed to the short packing distance between the P3HT and non-fullerene acceptor molecules at the donor-acceptor interfaces (DA distance), which is a rarely studied interfacial structural property, highly important in determining the decay rate of CT states; 3) the lower voltage loss in the state-of-the-art P3HT solar cell based on the 2,2'- ((12,13-bis(2-butyldecyl)-3,9diundecyl-12,13-dihydro-[1,2,5]-thiadiazolo[3,4-e]thieno[2'',3'':4',5']thieno[2',3':4,5]p-yrolo[3,2-g]thieno [2',3':4,5]thieno [3,2b]indole-2,10-diyl)bis(methanelylidene))bis(5,6-dichloro-1H-indene-1,3(2H)-dion-e) (ZY-4Cl) acceptor could be associated with the better alignment of the energy levels of the active materials and the longer DA distance, compared to those based on the commonly used acceptors. However, the DA distance was still very short, limiting the device voltage. Thus, improving the performance of the P3HT based solar cells requires a further increase in the DA distance. Our findings are expected to pave the way for breaking the performance bottleneck of the P3HT based solar cells.