The softening behaviour during non-isothermal annealing of a cold-rolled Al-Mn-Fe-Si model alloy was studied as a function of the state of microchemistry, in terms of the solute level of Mn, size and spatial distribution of the Mn-bearing dispersoids, as well as their temporal evolution. Microchemistry significantly affects the recrystallization microstructure, crystallographic texture as well as the mechanical property of the investigated alloy after non-isothermal annealing. The nucleation and growth of grains with different orientations are strongly dependent on both annealing temperature and microchemistry, in that pre-existing dispersoids have a less profound effect on retarding recrystallization than dispersoids forming concurrently during back-annealing. Strong concurrent precipitation suppresses nucleation and retards recrystallization, which finally leads to a coarse and pan-cake shaped grain structure, accompanied by strong P {011}<566> and/or M {113}<110> texture components and a relatively weaker ND-rotated cube {001}<310> component. A refined grain structure with medium strength P and cube {001}<100> components is obtained when the pre-existing dispersoids are coarser and fewer, and concurrent precipitation is limited. Porientated grains are less affected by second phase particles and experience a growth advantage at low annealing temperatures (<350°C), while M-orientated grains appear at higher temperatures. The intensity of the P texture does not necessarily increase with increasing supersaturation of Mn as observed during isothermal annealing, whereas the level of supersaturated Mn promotes the strength of the M texture. The mechanisms behind are discussed.