Thermal conductivity of single-crystal materials is crucial in the fields of lasers and nonlinear optics. Understanding the physical mechanism of thermal conductivity in such systems is therefore of great importance. In the present work, first principles calculations were employed to study the thermal conductivity of the infrared nonlinear optical materials, CdSiP2 and AgGaS2. These compounds crystallize in similar structures but with an order-of-magnitude difference in thermal conductivity. The average Grüneisen parameters are −0.8 and −2.6 for CdSiP2 and AgGaS2, respectively; these values are indicative of the soft-mode phenomenon of acoustic phonons. Crystal structures are considered unstable at low temperature through the whole Brillouin zone, especially in the region from K-point X to Γ. Acoustic phonon anharmonicity is concluded to be the main factor that determines the magnitude of thermal conductivity.