The influence of a prealloying process on the formation of MoSe2 and thus on the performance of Cu2ZnSnSe4 (CZTSe) solar cells is investigated using sputtering deposition and post‐annealing approaches. The dense alloy layer, which is made by a low‐temperature prealloying process, acts as a temporary Se diffusion barrier during a subsequent high‐temperature selenization process. The formation of thick interfacial MoSe2 can be suppressed effectively by this temporary barrier, cooperating with subsequent quick formation of compact CZTSe layer. The thickness of interfacial MoSe2 layer in CZTSe solar cells can be tailored by adjusting the preannealing process during selenization. As a consequence, the series resistance of CZTSe solar cells is reduced to a low level (≈0.6 Ω cm2), and the performance of CZTSe solar cells is improved significantly. A CZTSe solar cell with efficiency of 8.7% is fabricated.