We address the problem of damage identification in complex civil infrastructure with an integrative modular Bayesian framework. The proposed approach uses multiple response Gaussian processes to build an informative yet computationally affordable probabilistic model, which detects damage through inverse updating. Performance of structural components associated with parameters of the developed model was quantified with a damage metric. Particular emphasis is given to environmental and operational effects, parametric uncertainty and model discrepancy. Additional difficulties due to usage of costly physics-based models and noisy observations are also taken into account. The framework has been used to identify a reduction of a simulated cantilever beam elastic modulus, and anomalous features in main/stay cables and bearings of the Tamar bridge. In the latter case study, displacements, natural frequencies, temperature and traffic monitored throughout one year were used to form a reference baseline, which was compared against a current state, based on one month worth of data. Results suggest that the proposed approach can identify damage with a small error margin, even under the presence of model discrepancy. However, if parameters are sensitive to environmental/operational effects, as observed for the Tamar bridge stay cables, false alarms might occur. Validation with monitored data is also highlighted and supports the feasibility of the proposed approach.