Maintaining good friction performance of highway pavement is important for road safety. The friction is affected by many factors, and the present study investigates the effect of the compactness on the texture and friction of asphalt concrete during the polishing process. Two three-dimensional (3D) texture parameters and the mean texture depth (MTD) were used to characterize the surface texture of AC-13 asphalt concrete. The differences of surface texture are then being analyzed among the pavement in the field, rutting slabs with 97% compactness (RS-97), rutting slabs with 100% compactness (RS-100), and rutting slabs with 103% (RS-103). The rutting slabs were polished by a circular vehicle simulator (CVS). The 3D surface topography, British pendulum number (BPN), and MTD were obtained during the polishing process. Test results show that the surface of the rutting slab can be smoother as the compactness increased from 97% to 103%. During the whole polishing process, the rutting slab with smaller compactness had higher value of the MTD. The impact of compactness on the BPN is insignificant during the polishing process, but rutting slabs with smaller compactness had better friction at high speed as the result of the higher MTD.