Manipulation of phonon transport is the key to optimizing the overall energy conversion efficiency of thermoelectric materials. In this work, we demonstrate that the antibonding hybridization resulting from elemental substitution can weaken the interatomic interactions, which in turn enhances the structural anharmonicity and hinders the heat conduction of layered ABTe 3 (ABSe 3 ) (A = Al, Ga, In, As, and Sb; B = Si and Ge) compounds. It is revealed that the filled antibonded p-d state of GaSiTe 3 (GaSiSe 3 ) originates from the mixture of Ga-3d and Te-5p (Se-4p) orbitals, whereas the outer As-s electrons hybridize with the Te (Se)-p electrons to form antibonding states in AsSiTe 3 (AsSiSe 3 ). Consequently, the room temperature lattice thermal conductivity (κ L ) of AlGeTe 3 (AlGeSe 3 ) is reduced from 3.27 (3.86) W/mK to 0.62 (1.47) W/mK for GaSiTe 3 (GaSiSe 3 ) and 1.91 (1.74) W/mK for AsSiTe 3 (AsSiSe 3 ) in spite of their similar atomic masses and crystal structures. Based on these findings, we propose candidates of InGeTe 3 and SbGeTe 3 to realize potentially high thermoelectric performance by rationally incorporating heavy weight elements but still maintaining weak atomic binding interactions. Due to the low κ L of 0.26 (0.6) W/mK, a high n-type (p-type) ZT value of 2.18 (1.08) at 750 (650) K is finally captured in InGeTe 3 (SbGeTe 3 ). Our results not only provide insights to understand the relationship between chemical bonding and lattice thermal conductivity but also offer an approach to design and discover materials with expected thermal transport and thermoelectric properties.