The accuracy of physical property predictions using classical molecular dynamics simulations is determined by the quality of the empirical interatomic potentials (EIPs). We introduce a training approach for EIPs, based on direct comparisons of the second-and third-order interatomic force constants (IFCs) between EIP and density functional theory (DFT) calculations. This work's unique aspect is the utilization of irreducible derivatives (IDs) of the total energy, which leverage on the symmetry of the crystalline structure and provide a minimal representation of the IFCs. Our approach is tailored toward accurate predictions of thermal conductivity, thus requiring knowledge of both harmonic and anharmonic IFCs by matching second-and third-order displacement derivatives, whereas second-order strain derivatives are needed for determining the elastic constants. We demonstrate this approach as an efficient and robust manner in which to train EIPs for predicting phonons and related properties, by optimizing parameters of an embedded-atom method potential for ThO2, which is used as a model system for fluorite oxides. Our ID-trained EIP provides thermophysical properties in great agreement with DFT, and outperforms previously widely utilized EIP for ThO2 in phonon dispersion and thermal conductivity calculations. It also provides reasonable estimates of thermal expansion and the formation energies of simple defects.