Low-temperature crystal structure of the ErxDy1−xAl2 alloys with x = 0.45, 0.67, 0.90 was examined using temperature-dependent powder X-ray diffraction. The Er-rich sample, Er0.9Dy0.1Al2, exhibits a rhombohedral distortion associated with the magnetic ordering that occurs around 20 K. The rhombohedral distortion is suppressed in Er0.67Dy0.33Al2, while a weak low-temperature tetragonal distortion is observed in Er0.45Dy0.55Al2. The mean-field theory supports the correlation between the type of structural distortion and the variable easy magnetization axis in ErxDy1−xAl2 intermetallics.