Local villagers are regarded as the main part of tourism development in the ethnic village, their participation in tourism poverty alleviation has a vital impact on sustainable tourism development and poverty reduction. Based on the planned behavior theory and capacity approach theory, we investigated the influence of tourism empowerment on ethnic village residents' behavior participating in tourism poverty alleviation, especially focusing on the mediating effect of participation willingness and the moderating effect of participation ability. We took Zenlei Village in Sandu County of Guizhou Province in China as research subjects and analyzed 239 valid samples through the structural equation model and hierarchical regression. The results indicate that: (a) Tourism empowerment has a remarkable positive effect on participation behavior. (b) Participation willingness plays a partial mediating role between tourism empowerment and participation behavior. (c) Participation ability positively moderates the positive relationships between tourism economic empowerment, tourism psychological empowerment, and participation willingness. That is, the positive relationships between tourism economic empowerment, tourism psychological empowerment, and participation willingness would be stronger when villagers have higher participation ability. Whereas, participation ability has not yet played a moderating role between tourism social empowerment, tourism political empowerment, and participation willingness.