Two-dimensional silicon phononic crystals have attracted extensive research interest for thermoelectric applications due to their reproducible low thermal conductivity and sufficiently good electrical properties. For thermoelectric devices in high-temperature environment, the coherent phonon interference is strongly suppressed; therefore phonon transport in the incoherent regime is critically important for manipulating their thermal conductivity. On the basis of perturbation theory, we present herein a novel phonon scattering process from the perspective of bond order imperfections in the surface skin of nanostructures. We incorporate this strongly frequency-dependent scattering rate into the phonon Boltzmann transport equation and reproduce the ultra low thermal conductivity of holey silicon nanostructures. We reveal that the remarkable reduction of thermal conductivity originates not only from the impediment of low-frequency phonons by normal boundary scattering, but also from the severe suppression of high-frequency phonons by surface bond order imperfections scattering. Our theory not only reveals the role of the holey surface on the phonon transport, but also provide a computation tool for thermal conductivity modification in nanostructures through surface engineering.