Rockmass in deep mining is highly susceptible to large-scale collapses under high stress and blast-induced disturbances, leading to casualties and economic losses. To investigate the evolution characteristics of goaf instability and the types of seismic sources that induce instability, an experiment on goaf instability was designed under uniaxial compression conditions based on actual mining operations. The entire experimental process was monitored using digital image correlation analysis and acoustic emission monitoring. By calculating the digital speckle field on the surface of the rock specimen during the experiment, the evolution characteristics of the deformation and strain fields from the beginning of loading to complete failure were analyzed. The study explored the dynamic behavior of cracks from initiation to propagation and eventually inducing large-scale collapse. The results show that the instability process of the goaf begins with the formation of tensile cracks. As stress increases, shear cracks occur in the specimen, leading to macroscopic failure. Furthermore, based on the differences in overall microfracture types measured by RA-AF characteristic parameters during specimen failure, large amplitude acoustic emission events corresponding to the formation of dominant macroscopic cracks were selected, and the focal mechanisms of these events were inverted. The results indicate that shear failure sources are significantly more prevalent than tensile failure sources in acoustic emission events leading to goaf instability. These findings can provide useful guidance for the support design and the prevention and control of rockmass instability disasters.