In this study polyester‐based thermoplastic polyurethane (TPU) samples were annealed at 110°C for 0, 8, 16, and 24 h. To elucidate the relationship between the hard and soft phase separation and the shape memory properties, the samples were characterized within a rheometer, by selecting well defined thermal cycles. The results showed an enhancement of the shape recovery ratio (Rr ~ 65%) for all the annealed samples, when compared to the non‐annealed TPU (Rr ~ 60%). This behavior was attributed to the modifications of hard‐soft domain morphology occurring during the thermal annealing treatment, as it was shown both by differential scanning calorimetry with the shift in thermal transitions toward higher temperature, due the formation of new short‐range ordered hard domains and infrared spectroscopy, with the increase in degree of phase separation. Analyzing small‐angle X‐ray scattering, the decrease of the invariant also confirmed the enhancement of short‐range domains. The formation of these new short‐range domains acted as crosslinking points in TPU's morphology leading to an increase in stiffness, presented by higher Young and storage modulus. Based on these results, a model mechanism is proposed to correlate the morphology and structure of the annealed materials with their shape memory effect.