Using a three-dimensional classical ensemble method, we investigated the nonsequential double ionization (NSDI) of xenon atoms from the near infrared to the mid-infrared in spatially inhomogeneous laser fields, and compared with results from spatially homogeneous laser fields. The results show that the NSDI probability curves are similar between spatially inhomogeneous laser field and spatially homogeneous laser field at short wavelength. With the laser wavelength increasing, at the high intensities NSDI is more and more suppressed for spatially inhomogeneous laser field. Compared with spatially homogeneous laser fields, the final emission angle of two electrons from NSDI exhibits a much stronger correlated characteristic in spatially inhomogeneous fields, especially at longer laser wavelengths, the final emission angle of two electrons is almost all concentrated around 0° meaning that the two electrons are always emitted into the same direction parallelly. Moreover, effective recollision of the NSDI is always dominated by first return of the first electron from the near infrared to the mid-infrared inhomogeneous laser fields, however, the transition from the first return dominance to the second return dominance occurs in the spatially homogeneous laser fields. Further, we reveal more details of the ultrafast dynamics of the correlated electrons in the spatially inhomogeneous laser fields by back-tracing the classical trajectories of NSDI.