The optical reflection characteristics of asphalt pavement have an important influence on road-lighting design, and the macrotexture and microtexture of asphalt pavement significantly affect its reflection characteristics. To investigate the impact of texture parameters on the retroreflection coefficient of asphalt pavement, the texture indices of rutted plate specimens and field asphalt pavement were obtained by a pavement texture tester, including the macrotexture surface area (S1), microtexture surface area (S2), macrotexture distribution density (D1), microtexture distribution density (D2), root mean square slope (Δq), skewness (Rsk), and steepness (Rku). The corresponding retroreflective coefficient RL was measured by using a retroreflectometer. In the laboratory experiments, rutted specimens of AC-13, SMA-13, and OGFC-13 asphalt mixtures were formed. The changes in texture parameters and the retroreflection coefficient of rutting specimens before and after rolling were studied, and a factor-influence model between macro- and microtexture parameters and RL was established, along with correlation models of the texture index and RL. The results show that after the rutting test, S1, S2, D1, D2, Δq, and Rku decreased, Rsk increased, and RL increased. In the single-factor model, the parameters could be used to characterize RL with high prediction accuracy, whereas for the onsite measurements, the parameters Δq, Rsk, and Rku could well characterize RL. The nonlinear model established, based on the BP neural network algorithm, improved the prediction accuracy. This research provides ideas for optimizing the reflection characteristics of asphalt pavement and a decision-making basis for road-lighting design.