As a non-invasive and remote sensor, the Laser Doppler Vibrometer (LDV) has found a broad spectrum of applications in various areas such as civil engineering, biomedical engineering, and even security and restoration within art museums. LDV is an ideal sensor to detect threats earlier and provide better protection to society, which is of utmost importance to military and law enforcement institutions. However, the use of LDV in situational surveillance, in particular vehicle classification, is still in its infancy due to the lack of systematic investigations on its behavioral properties. In this work, as a result of the pilot project initiated by Air Force Research Laboratory, the innate features of LDV data from many vehicles are examined, beginning with an investigation of feature differences compared to human speech signals. A spectral tone-pitch vibration indexing scheme is developed to capture the engine’s periodic vibrations and the associated fundamental frequencies over the vehicles’ surface. A two-layer feed-forward neural network with 20 intermediate neurons is employed to classify vehicles’ engines based on their spectral tone-pitch indices. The classification results using the proposed approach over the complete LDV dataset collected by the project are exceedingly encouraging; consistently higher than 96% accuracies are attained for all four types of engines collected from this project.