Instability, as a potential mechanism for vortex flow control, is manifested typically in the phenomenon of wingtip vortex wandering. In this paper, the evolution of wingtip vortex generated by a NACA0015 rectangular wing within six chord lengths of its wake at Re c = 2.1 × 10 5-3.5 × 10 5 and AoA = 4°-10° is investigated by stereo particle image velocimetry experiment. The streamwise evolution of vortex wandering is clearly captured, which demonstrates strong anisotropy. The amplitude of vortex wandering increases under all conditions, which exhibits greater growth rates in the condition of larger AoA. To account for such phenomenon, spatial linear stability analysis is performed on the base flow obtained by SPIV experiment to investigate quantitatively the amplification of small disturbance and disturbance modes at the experiment conditions. It is discovered that the wingtip vortex in the experiment is marginally stable under all conditions. Furthermore, the spatial growth rates of instability by AoA correspond well with that of wandering amplitude, indicating that the quick enlargement of the amplitude of wandering at larger AoA condition is caused by larger spatial growth rate of disturbance. In addition, the least-stable disturbance mode by LSA reveals a non-zero directed velocity perturbation in the vortex core, which rotates periodically. Such perturbation developing in both time and space, which is stirred by vortex instability, leads to the behavior of wingtip vortex wandering.