Despite the growing interest in nonlinear devices and components for light by light control in the terahertz range, there is a shortage of such materials and media used for these purposes. Here, we present the calculated values of low-inertia nonlinear refractive index coefficient for electro-optical crystals used in THz time-domain spectroscopy systems such as ZnSe, ZnTe, CdTe, GaP, and LiNbO3. The medium parameters affecting the cubic nonlinearity of the vibrational nature increase in the range of 0.5–1 THz have been determined. Comparison of theoretical calculations with known experimental results confirm the theoretical model as well as our analysis of media parameter influence on the cubic nonlinearity. In terms of applications, results obtained open up new perspectives for studying various materials in the THz frequency range.