Strain relaxation mechanisms in AlGaN based light emitting diodes emitting in the ultraviolet B spectral range (UVB‐LEDs) grown on different AlN/sapphire templates are analyzed by combining in situ reflectivity and curvature data with transmission electron microscopy. In particular, the impact of dislocation density, surface morphology, and lattice constant of the AlN/sapphire templates is studied. For nonannealed AlN/templates with threading dislocation densities (TDDs) of 4 × 109 and 3 × 109 cm−2 and different surface morphologies strain relaxation takes place mostly by conventional ways, such as inclination of threading dislocation lines and formation of horizontal dislocation bands. In contrast, a TDD reduction down to 1 × 109 cm−2 as well as a reduction of the lattice constant of high temperature annealed AlN template leads to drastic changes in the structure of subsequently grown AlGaN layers, e.g., to transformation to helical dislocations and enhanced surface enlargement by formation of macrofacets. For the growth of strongly compressively strained AlGaN layers for UVB‐LEDs the relaxation mechanism is strongly influenced by the absolute values of TDD and the lattice constant of the AlN templates and is less influenced by their surface morphology.