In this study, the effectiveness of local vibration testing using a portable scanner and machine learning for non-destructive inspection of concrete structures was examined. Using a portable vibrator and laser vibrometer, local throughthickness vibration tests were conducted on specimens of concrete beams and a railway track containing voids. Comparisons between frequency response functions measured over intact and void regions within structural concrete showed wave damping around the void. Moreover, a large amount of frequency response functions was obtained with the laser vibrometer in the scanning testing. With the measured data, Support Vector Machine (Kernel method) was used for detecting voids within the beams and beneath the railway track slab. Through its analysis of frequency response functions, the measured data of the intact or void condition was classified with a percent accuracy of 70%. This result indicates promising usage of the proposed method: utilizing a portable vibrator, laser vibrometer, and machine learning for non-destructive inspection of concrete structures.