Roughness is a comprehensive assessment indicator of pavement performance. Prediction of pavement roughness exhibits great difficulties by using traditional methods such as mechanistic-empirical method and regression method. Considering the fact that the value of international roughness index (IRI) varies in a fluctuant manner, in this paper, a new gray model based method is proposed to predict the roughness of pavement. The proposed method adopts GM(1,1|sin) model as the prediction model. In GM(1,1|sin) model, a sinusoidal term is added into GM(1,1) model, making it can fit fluctuant data more precisely than GM(1,1) model. A particle swarm optimization (PSO) algorithm is used to select the optimal parameter of GM(1,1|sin) model. Experimental results demonstrate its effectiveness of the proposed method. Furthermore, the proposed method only uses the history IRI data in prediction and leads to a large savings of collecting pavement condition data.