Spectral emissivity is an essential and sensitive parameter to characterize the radiative capacity of the solid surface in scientific and engineering applications, which would be non-negligibly affected by surface morphology. However, there is a lack of assessment of the effect of roughness on emissivity and a straightforward method for estimating the emissivity of rough surfaces. This paper established an estimating method based on constructing random rough surfaces to predict rough surface (Geometric region) emissivity for metal solids. Based on this method, the emissivity of ideal gray and non-gray body surfaces was calculated and analyzed. The calculated and measured spectral emissivities of GH3044, K465, DD6, and TC4 alloys with different roughness were compared. The results show that the emissivity increases with the roughness degree, and the enhancement effect weakens with the increase of roughness or emissivity due to the existing limit (emissivity ε = 1.0). At the same time, the roughness would not change the overall spectral distribution characteristics but may attenuate the local features of the spectral emissivity. The estimated results are in good agreement with the experimental data for the above alloys’ rough surfaces. This study provides a new reliable approach to obtaining the spectral emissivity of rough surfaces. This approach is especially beneficial for measuring objects in extreme environments where emissivity is difficult to obtain. Meanwhile, this study promotes an understanding of surface morphology’s effect mechanism on emissivity.