The optical intensity transmitted through a random pattern of subwavelength holes in a metal film exhibits a speckle pattern. We study the variation of this speckle pattern as a function of wavelength. We find that the resulting speckle correlation function (SCF) separates into a wavelength-dependent part and a wavelength-independent background. The wavelength dependence is caused by surface plasmons excited at one hole and coupled out at another hole, while the constant background originates from light transmitted directly through the holes. By analyzing the SCF for a set of samples of varying hole density, we find the propagation length of the surface plasmons and the scattering losses induced by the holes.