The aim of this work is to study the structural changes that take place during the heat setting of simultaneously and sequentially biaxially stretched polyethylene terephthalate films. The rate of structural rearrangement, as detected by real‐time birefringence and off‐line Raman spectroscopy, DSC thermal analysis and X‐rays, was found to be dependent on the state of the orientation and crystallinity attained by the samples during the stretching step. This suggested that the annealing behavior can be categorized into three regimes according to the birefringence behavior of the samples. At low deformations, the films not crystallized by strain exhibit complete birefringence relaxation without showing any sign of crystallization. The intermediate ratio samples that were stretched to the onset of the strain‐induced crystallization exhibited the highest levels of birefringence increase after a short relaxation. The samples stretched well above the onset of the strain‐induced crystallization exhibited intermediate level of birefringence and crystallinity increase. Therefore, the classification of the three regimes was found to be related to the onset of the strain‐induced crystallization during the stretching step. POLYM. ENG. SCI., 57:550–562, 2017. © 2016 Society of Plastics Engineers