The effect of varying temperatures is one of the most important challenges of vibration-based damage identification due to its bigger effects on the structural response than the damage itself. This study presents a methodology incorporating the autoregressive (AR) time series model with two-step artificial neural networks (ANNs) to identify damage under temperature variations. AR coefficients, which are extracted by fitting the AR models to acceleration responses, are however sensitive to temperature changes, resulting in false diagnoses. Thus, two-step ANN models with the inputs of difference in AR coefficients are utilized to compensate the detrimental temperature variations. Finite element (FE) models of a steel-braced frame structure, simulating several damage scenarios with different damage locations and severities at fluctuating temperatures, are used to verify the effectiveness and reliability of this approach. Numerical results indicate that the proposed approach could successfully recognize, locate, and quantify damage by using output-only vibration and temperature data regardless of varying temperatures and noise perturbations.