Optical nonlinearities in the undoped As20S80, low doped P2As20S78 and Sn1As20S79 chacogenide glasses are investigated by using Z-scan method. These experiments show that at 1064 nm the figure of merit (FOM) for As20S80 is ∼1.02, while for Sn1As20S79 increases to ∼1.42, and for P2As20S78 decreases to ∼0.83. These resulted data indicate the addition of Sn in As20S80 system chalcogenide glass can enhance FOM due to creating narrow energy gaps. In addition, the self-phase modulation (SPM) width experiment for Sn1As20S79 strip waveguide displays that the full width half maximum (FWHM) of spectral width increases approximately 0.8 nm under the induction of bandgap light, meaning that the bandgap light can induce to enhance its optical nonlinearity with the nonlinear refractive index of n2≅5.27×10−14 cm2/W.