For vertical axis wind turbines (VAWTs), the increase of the incoming wind speed higher than the rated value will make the tip speed ratio (TSR) lower and lower, resulting in the blade fatigue load becoming more and more severe and the power coefficient weakening gradually. This paper explores whether varying the pitch with the TSR decrease is necessary for improving the power coefficient and reducing the fatigue load. Specifically, the pitch angle effect on the power coefficient and fatigue load of a VAWT at different TSRs was studied by the computational fluid dynamics method. The results show that the optimal pitch angle in terms of the power coefficient varies with the TSR, which means that varying the pitch with the TSR decrease can improve the power coefficient. Meanwhile, the principle to guide the pitch variation is to avoid flow separation in the downwind zone and minimize the angles of attack (AoAs) in the upwind zone. At the lowest TSR of 1.7 in the present work, varying the pitch from the optimal one in terms of the power coefficient reduced the blade normal force amplitude significantly, which is mainly attributed to avoiding the vortex–blade encounter and minimizing the AoAs in the downwind zone. The vortex–blade encounter at the lowest TSR is an important phenomenon related to the variation of the blade torque and blade normal force and will weaken and disappear with the pitch angle increase.