Time-varying environmental and operational conditions such as temperature and external loading may produce an adverse effect on damage detection with the structure exposed to these changes. In fact, these effects can often mask more subtle structural changes caused by damage. Therefore, in order to achieve successful structural health monitoring goal, a new data normalization technique based on the improved restoring force model (IRFM) is proposed in this paper to distinguish the effect of damage from those caused by environmental and operational variations. Firstly, a special training data set, whose IRFM coefficients are closest to the IRFM coefficients from the testing data, is determined and selected by a Euclidean distance measure. The IRFM coefficients of this training data are then applied to calculate the residual errors of this special training data and the testing data for the respective IRFM approximations to these measured time histories. Finally, the ratio of the variance of the residual errors is defined as the damage-sensitive feature and used in the outlier detection process. The usefulness of the proposed approach is demonstrated using an experimental study tested at Los Alamos National Laboratory.