Context. The tangential YORP effect is a radiation pressure torque produced by asymmetric thermal emission by structures on the asteroid surface. Previous works considered these structures to be boulders of different shapes lying on the surface of the asteroid. Aims. We study the tangential YORP produced by the rough interface of the asteroid's regolith. Methods. We created an approximate analytic theory of heat conduction on a slightly non-flat sinusoidal surface. We analyzed the published data on the small-scale shape of the asteroid (162173) Ryugu and estimated its tangential YORP due to the surface roughness. Results. We derive an analytic formula that expresses the tangential YORP of a sinusoidal surface in terms of its geometric and thermal properties. The tangential YORP is highest at the thermal parameter on the order of unity and for shape irregularities on the order of the thermal wavelength. Application of this equation to Ryugu predicts a tangential YORP that is 5-70 times greater than its normal YORP effect. Conclusions. The contribution of the small-scale regolith roughness to the YORP effect of the asteroid can be comparable to the normal YORP and the tangential YORP produced by boulders. The same theory can describe the roughness of the asteroid boulders, thus adding a new term to the previously considered the tangential YORP created by boulders.