In a rarefied gas with a non-uniform temperature field, one phenomenon that arises is the tangential Knudsen force. Various researches have investigated the tangential Knudsen force but have been limited to specific cases. In this study, we investigated the mechanism of the thermally induced tangential Knudsen force, using theoretical analysis under fully diffusive conditions and for a range of Knudsen numbers. Specifically, we formulated a theoretical expression to describe the tangential Knudsen stress by considering the two kinds of -momentum fluxes transferred on a surface of interest. One is brought by molecules directly coming from the other surface without experiencing intermolecular collisions and the other is brought by molecules coming from the bulk region after experiencing intermolecular collisions there. As reference, we used a channel where the lower surface is a hot ratchet structure and the upper surface is a flat cold object. The tangential Knudsen force on the object obtained by the theoretical analysis was compared with the results from our previous work where we performed numerical experiments by the direct simulation Monte Carlo (DSMC) method. Based on the comparison, it is found that the tangential Knudsen force is caused by three mechanisms. First is the contribution of impinging molecules coming from the other surface with different temperature. Second is the contribution of viscous effect of thermally driven flows. While the third is the contribution of thermal stress, which is noticeable in small Knudsen numbers.