In this work, based on nonlocal heat effect and Fresnel diffraction theory, the characteristics of closed Z-scan in materials with a large nonlinear phase shift have been studied and a formula for Z-scan transmittance is derived. Numerical computations show that the shapes of Z-scan curves are greatly affected by the values of thermal nonlinear phase shift. The Z-scan behavior is compared to the conventional thermal Z-scan theories, including the thermal thin lens and aberrant thermal lens model. We have used the Z-scan models to determine the nonlinear refraction of gold nanoparticle (AuNP) dispersed in cyclohexanone under irradiation of a low power CW laser at 532 nm. The Z-scan theory based on the conventional models does not fit the experimental data well. It is found that peak-to-valley features of the AuNPs colloid Z-scan curve behave the same as the derived Z-scan theory for large thermal nonlinear phase shift. This allows us to measure nonlinear refractive index up to −1.0 × 10 −6 cm 2 /W in the tested samples. The optical limiting performance of the colloidal AuNPs is also investigated for device application. The results show that the optical limiting threshold of the AuNPs can be improved by making an appropriate choice of design geometry and increasing the AuNPs concentration of the colloids.Keywords: Z-scan theory; nonlocal heat effect; gold nanoparticles colloid; nonlinear refractive index; thermo-optic coefficient; optical limiter. 1550014-1 J. Nonlinear Optic. Phys. Mat. 2015.24. Downloaded from www.worldscientific.com by NANYANG TECHNOLOGICAL UNIVERSITY on 08/22/15. For personal use only.