For a wide industrial application of magnesium alloys, a method for imparting high damping properties while maintaining mechanical properties is required. Controlling the crystallographic texture seems to be useful, because dislocations are known to have a significant influence on the damping characteristics of magnesium alloys. In addition, textures are affected by the microstructure and texture variation when the deformation or annealing is applied. However, there were less reports about their effect on damping capacity. Therefore, the effect of twinning and annealing, which can affect the recrystallization, were investigated in this study. An AZ31 alloy was hot rolled at 673 K with a reduction ratio of 10% and 50%, and then annealed at 673 K and 723 K for 0.5, 1, 2, and 3 h, respectively. SEM-EBSD was used to examine the microstructure and texture. In addition, each specimen’s hardness and internal friction were contemporarily measured. As a result, hot rolling produced tensile twins and their fraction increased with internal friction when the reduction ratio increased. Due to annealing, a discontinuous type of static recrystallization occurred within the twinning grains, and was highly activated along with the increasing annealing temperature and the fraction of twinning. In the samples annealed at 723 K, the internal friction continuously increased over the annealing time, whereas in the samples annealed at 673 K, the decrease in dislocation density was delayed while the internal friction showed a relatively low value.