We present a systematic study of a mobile impurity immersed in a three-dimensional Fermi sea of fermions at finite temperature, by using the standard non-self-consistent many-body T -matrix theory that is equivalent to a finite-temperature variational approach with the inclusion of one-particlehole excitation. The impurity spectral function is determined in the real-frequency domain, avoiding any potential errors due to the numerical analytic continuation in previous T -matrix calculations and the small spectral broadening parameter used in variational calculations. In the weak-coupling limit, we find that the quasiparticle decay rate of both attractive and repulsive polarons does not increase significantly with increasing temperature, and therefore Fermi polarons may remain welldefined far above Fermi degeneracy. In contrast, near the unitary limit with strong coupling, the decay rate of Fermi polarons rapidly increase and the quasiparticle picture breaks down close to the Fermi temperature. We analyze in detail the recent ejection and injection radio-frequency (rf) spectroscopy measurements, performed at Massachusetts Institute of Technology (MIT) and at European Laboratory for Non-Linear Spectroscopy (LENS), respectively. We show that the momentum average of the spectral function, which is necessary to account for the observed rfspectroscopy, has a sizable contribution to the width of the quasiparticle peak in spectroscopy. As a result, the measured decay rate of Fermi polarons could be significantly larger than the calculated quasiparticle decay rate at zero momentum. By take this crucial contribution into account, we find that there is a reasonable agreement between theory and experiment for the lifetime of Fermi polarons in the strong-coupling regime, as long as they remain well-defined.