Research on sheath expansion is critical to the understanding of the dielectric recovery process in a vacuum interrupter after interruption of vacuum arcs. In this paper, we investigated how residual plasma affects breakdown in the sheath expansion period after the current zero. To simulate sheath expansion and breakdown, we developed a fully kinetic particle-in-cell Monte Carlo collision model with one spatial dimension and three velocity dimensions. The model accounted for various collisions, including ionization, excitation, elastic collisions, charge exchange, and momentum exchange, and we added an external circuit to the model to make the calculations self-consistent. The existence of metal vapor slowed the sheath expansion in the gap and caused high electric field formation in front of the cathode surface. The initial residual plasma, which was at sufficiently low density, seemed to have a limited impact on breakdown, and the metal vapor dominated the breakdown in this case. Additionally, the breakdown probability was sensitive to the initial plasma density if the value exceeded a specific threshold, and plasma at sufficiently high density could mean that breakdown would occur more easily. We found that if the simulation does not take the residual plasma into account, it could overestimate the critical value of the metal vapor density, which is always used to describe the boundary of breakdown after interruption of vacuum arcs. We discussed the breakdown mechanism in sheath expansion, and the breakdown is determined by a combination of metal vapor, residual plasma, and the electric field in front of the cathode surface.