Composite materials were prepared by the hot pressing method using poplar seed fibers and polycaprolactone (PCL) as the raw materials to solve the problems related to the recycling of waste fibers. The effects of mass fraction of poplar seed fibers, the volume density, and thickness on the sound absorption performance of the resulting composite materials were studied. The sound absorption coefficient curves of the composite material were obtained by the acoustic impedance transfer function method. The sound absorption coefficient of the composite material that was prepared under the optimal process conditions was higher than 0.7, and the effective sound absorption frequency band was wide. According to the box counting dimension method, which is based on the fractal theory, the fractal dimensions of the composite materials were calculated while using the Matlab program. The relationships between the fractal dimensions and the volume densities, mass fractions of poplar seed fibers, and thicknesses of the composite materials were also analyzed. Subsequently, the quantitative relationship between the fractal dimension and the sound absorption property parameters of the composite material was established in order to provide a theoretical basis for the design of the sound absorption composite material.