The application of thinner cadmium sulfide (CdS) window layer is a feasible approach to improve the performance of cadmium telluride (CdTe) thin film solar cells. However, the reduction of compactness and continuity of thinner CdS always deteriorates the device performance. In this work, transparent Al2O3 films with different thicknesses, deposited by using atomic layer deposition (ALD), were utilized as buffer layers between the front electrode transparent conductive oxide (TCO) and CdS layers to solve this problem, and then, thin-film solar cells with a structure of TCO/Al2O3/CdS/CdTe/BC/Ni were fabricated. The characteristics of the ALD-Al2O3 films were studied by UV–visible transmittance spectrum, Raman spectroscopy, and atomic force microscopy (AFM). The light and dark J–V performances of solar cells were also measured by specific instrumentations. The transmittance measurement conducted on the TCO/Al2O3 films verified that the transmittance of TCO/Al2O3 were comparable to that of single TCO layer, meaning that no extra absorption loss occurred when Al2O3 buffer layers were introduced into cells. Furthermore, due to the advantages of the ALD method, the ALD-Al2O3 buffer layers formed an extremely continuous and uniform coverage on the substrates to effectively fill and block the tiny leakage channels in CdS/CdTe polycrystalline films and improve the characteristics of the interface between TCO and CdS. However, as the thickness of alumina increased, the negative effects of cells were gradually exposed, especially the increase of the series resistance (Rs) and the more serious “roll-over” phenomenon. Finally, the cell conversion efficiency (η) of more than 13.0% accompanied by optimized uniformity performances was successfully achieved corresponding to the 10 nm thick ALD-Al2O3 thin film.