In the early stage of atherosclerosis, the luminal surface of the arterial wall becomes rough. Methods for distinguishing between the reflected and backscattered components in the ultrasonic echo from the arterial wall has the potential to be used as a method for assessment of the roughness of the arterial wall. In this study, we proposed a method to distinguish between the reflected and backscattered components using a technique based on plane wave compounding. This method was evaluated by experiments using planar phantoms with rough surfaces made of polyurethane rubber. The coefficient of variation calculated from the mean value of the reflection component and the standard deviation of the backscattering component was proportional to the roughness of the rubber phantom. This result shows the potential usefulness of this method for analyzing surface roughness of the arterial wall.