GeSb2Te4 (GST124), one of the well‐known phase‐change materials for nonvolatile memory and rewritable optical storage, has been recently found to be promising thermoelectric materials with low lattice thermal conductivity and high electrical conductivity. However, its thermoelectric performance is greatly restricted by the excessively high hole concentration. Herein, the impact of a series of group IIIA (Al, Ga, In) and group VIA (S, Se) dopants on the electrical transport properties of polycrystalline GST124 has been studied. It is found that element sulfur (S) has the best doping efficiency because the GeS bonds are very strong and ionic that are beneficial for suppressing Ge vacancies to reduce the carrier concentration. Meanwhile, element indium (In) also shows decent doping efficiency because its ionic radius is close to the Ge ion and the InTe bonds have moderate bonding strength. Moreover, In doping introduces a resonant level in the valence band, leading to enhanced Seebeck coefficient and power factor. A high figure of merit (zT)of 0.73 at 700 K and an average zT of 0.48 over 300–750 K are obtained in Ge0.92In0.08Sb2Te4, which are 26% and 66% higher than pristine GST124. This study will advance the understanding and development of high‐performance GeSbTe‐based thermoelectric materials.