For infrastructures to be sustainable, it is essential to improve maintenance and management efficiency. Vibration-based monitoring methods are being investigated to improve the efficiency of infrastructure maintenance and management. In this paper, signals from acceleration sensors attached to vehicles traveling on bridges are processed. Methods have been proposed to individually estimate the modal parameters of bridges and road unevenness from vehicle vibrations. This study proposes a method to simultaneously estimate the mechanical parameters of the vehicle, bridge, and road unevenness with only a few constraints. Numerical validation examined the effect of introducing the Kalman filter on the accuracy of estimating the mechanical parameters of vehicles and bridges. In field tests, vehicle vibration, bridge vibration, and road unevenness were measured and verified, respectively. The road surface irregularities estimated by the proposed method were compared with the measured values, which were somewhat smaller than the measured values. Future studies are needed to improve the efficiency of vehicle vibration preprocessing and optimization methods and to establish a methodology for evaluating accuracy.