In recent years, a rapid bridge health monitoring technology has been developed using an instrumental moving vehicle. Using recorded vehicle vibration data, bridge frequencies are identified for bridge health monitoring or finite element model updating. Target bridge frequencies with significant amplitudes in the vehicle’s vibration frequency spectra are expected to be found. However, in the coupled vehicle–bridge interaction (VBI) system, bridge vibration-relevant vehicle dynamics might not be noticeable. The bridge frequency would be difficult to identify because of the potential influence of road roughness. To resolve this difficulty, a novel bridge frequency identification method is proposed to mitigate the negative effects of road roughness. First, theoretical derivations are done to ascertain the VBI system dynamic characteristics considering road surface roughness. Our findings showed that the road roughness-relevant vehicle dynamics are closely related with the traveling speed, whereas the bridge frequency remains approximately constant. Theoretical investigations indicated that cross-power spectra between vehicle dynamics at multiple moving speeds are effective to mitigate the negative effects of road roughness. Presumably, it is feasible to identify the target bridge frequency from the cross-power spectra. Both the dynamic characteristics of the VBI system and the effectiveness of the proposed method for bridge frequency identification were examined using finite element simulations and laboratory experiments. Compared to existing methods, the proposed method is widely applicable to real-world situations and difficulties.