Controlling both the film crystallinity of the active layer for better charge transport and the interdiffusion between donor and acceptor materials for optimal bicontinuous networks is essential in producing pseudo-bilayer polymer solar cells. In this work, we investigated the influence of a doping solution-processable small molecule with high carrier mobility, 5,11-bis(triethylsilylethynyl) anthradithiophene (TES-ADT), on the performance of pseudo-bilayer polymer solar cells made of an underlayer of poly(3-hexylthiophene) (P3HT) and an upper layer of [6,6]-phenyl-C61-butyric acid methyl ester (PCBM). By analysis of the X-ray diffraction and UV−vis absorbance spectra of P3HT:TES-ADT blend films it was demonstrated that the film crystallinity was enhanced by TES-ADT doping in the P3HT underlayer. The hole mobility extracted from the current density−voltage curves of hole-only devices based on P3HT:TES-ADT demonstrated an optimized value with proper TES-ADT doping and thermal annealing. An intermixed photoactive layer was observed for the annealed device, indicating the occurrence of interdiffusion with a large interfacial area. With improved film crystallinity and interdiffusion, the optimal device performance was obtained when 5% TES-ADT was blended with P3HT and a thermal annealing treatment at 150 °C for 1 min was conducted. At that optimal condition, the mean crystallite size was increased by 35%, and hence the enhancement of 8% and 14% in power conversion efficiency and short-circuit current density was observed, respectively.