Within the scope of concrete internal defect detection via laser Doppler vibrometry (LDV), the acquired signals frequently suffer from low signal-to-noise ratios (SNR) due to the heterogeneity of the concrete’s material properties and its rough surface structure. Consequently, these factors make the defect signal characteristics challenging to discern precisely. In response to this challenge, we propose an internal defect detection algorithm that incorporates local mean decomposition-singular value decomposition (LMD-SVD) and weighted spatial-spectral entropy (WSSE). Initially, the LDV vibration signal undergoes denoising via LMD and the SVD algorithms to reduce noise interference. Subsequently, the distribution of each frequency in the scan plane is analyzed utilizing the WSSE algorithm. Since the vibrational energy of the frequencies caused by the defect resonance is concentrated in the defect region, its energy distribution in the scan plane is non-uniform, resulting in a significant difference between the defect resonance frequencies’ SSE values and the other frequencies’ SSE values. This feature is used to estimate the resonant frequencies of internal defects. Ultimately, the defects are characterized based on the modal vibration patterns of the defect resonant frequencies. Tests were performed on two concrete blocks with simulated cavity defects, using an ultrasonic transducer as the excitation device to generate ultrasonic vibrations directly from the back of the blocks and applying an LDV as the acquisition device to collect vibration signals from their front sides. The results demonstrate the algorithm’s capacity to effectively pinpoint the information on the location and shape of shallow defects within the concrete, underscoring its practical significance for concrete internal defect detection in practical engineering scenarios.