Gully sidewall expansion is an important geomorphic natural hazard, and the expansion destroys a large extent of agricultural land in the loess regions every year. The main aim of this study was to identify the mechanisms behind gully sidewall expansion through a series of simulated rainfall experiments. The results show that land loss on the gentle slope was the result caused by the water and gravity erosions, and gravity erosion was the primary driving force. The correlation coefficient between the area of land loss on gentle slope and volume of gravity erosion on the gully sidewall was 0.93, and the correlation coefficient between the area of land loss on gentle slope and volume of water erosion was 0.71. The gravity erosion was the dominant impetus driving the change in slope gradient of the gully sidewall. The amount of gravity erosion in 17 of the 19 rainfall events causing a change greater than 5° in the slope gradient of the gully sidewall accounted for more than 50% of the total amount of sidewall erosion. Furthermore, the dynamic variation of the retreat rates for the gully shoulder line showed a similar trend to that of the total volume of sidewall erosion and exhibited an increase-decrease-increase tendency. The most significant factors affecting the change in slope gradient of the gully sidewall and retreat rate of the gully shoulder line were the rainfall duration and intensity, of which the sensitivity coefficients were 2.2 and 4.0, respectively. As a result, a combination of vegetation measures on the gentle slope, structural and ecological practices on the sidewall and powerful structural practices, e.g., check dams, on the gully floor, is preferred for sidewalls vulnerable to expansion.