Magnetic interaction with the gapless surface states in topological insulator (TI) has been predicted to give rise to a few exotic quantum phenomena. However, the effective magnetic doping of TI is still challenging in experiment. Using first-principles calculations, the magnetic doping properties (V, Cr, Mn and Fe) in three strong TIs (Bi2Se3, Bi2Te3 and Sb2Te3) are investigated. We find that for all three TIs the cation-site substitutional doping is most energetically favorable with anion-rich environment as the optimal growth condition. Further our results show that under the nominal doping concentration of 4%, Cr and Fe doped Bi2Se3, Bi2Te3, and Cr doped Sb2Te3 remain as insulator, while all TIs doped with V, Mn and Fe doped Sb2Te3 become metal. We also show that the magnetic interaction of Cr doped Bi2Se3 tends to be ferromagnetic, while Fe doped Bi2Se3 is likely to be antiferromagnetic. Finally, we estimate the magnetic coupling and the Curie temperature for the promising ferromagnetic insulator (Cr doped Bi2Se3) by Monte Carlo simulation. These findings may provide important guidance for the magnetism incorporation in TIs experimentally.