Twins, as one kind of crystal defects, are often been observed in various material systems. However, its effect on phonon scattering is much less investigated comparing to other defects. The difficulty in understanding the phonon scattering effect by twins lies in the inevitable presence of other defects (e.g., grain boundaries), which overshadow the intrinsic scattering effect by twins. In order to resolve the issue, three different kinds of InSb specimens with microscale grains, nanoscale grains, and nanoscale grains with nanotwins were prepared in this work. By comparing the thermal conductivities between different specimens, our results unambiguously show that phonons can indeed be scattered by twin boundaries. However, in a striking contrast to grain boundaries, phonon scattering effect by twin boundaries is noticeably weaker, which could be ascribed to the coherent characteristics of twin boundaries.