As urban infrastructure becomes denser, subway operations face numerous risks, including adjacent construction and structural damage deterioration. Integrated monitoring is crucial to ensure safety. This study presents a multitask vibration transformer (M-TVT) model, which utilizes an Expert-Gate network to manage task-related information sharing. The model achieves integrated identification for the type and spatial distribution of internal and external tunnel abnormal vibration responses, along with assessing structural damage degree. Meanwhile, a tunnel-train-soil coupling experiment is conducted to validate the efficacy of the M-TVT model. The results demonstrate that the M-TVT model with the Expert-Gate network maintains high accuracy and stability, efficiently shares parameter information among tasks and achieves improvement in accuracy compared to single-task models.