Some probability distributions have moments, and some do not. For example, the normal distribution has power moments of arbitrary order, but the Cauchy distribution does not have power moments. In this paper, by analogy with the renormalization method in quantum field theory, we suggest a renormalization scheme to remove the divergence in divergent moments. We establish more than one renormalization procedure to renormalize the same moment to prove that the renormalized moment is scheme-independent. The power moment is usually a positive-integer-power moment; in this paper, we introduce nonpositive-integer-power moments by a similar treatment of renormalization. An approach to calculating logarithmic moment from power moment is proposed, which can serve as a verification of the validity of the renormalization procedure. The renormalization schemes proposed are the zeta function scheme, the subtraction scheme, the weighted moment scheme, the cut-off scheme, the characteristic function scheme, the Mellin transformation scheme, and the power-logarithmic moment scheme. The probability distributions considered are the Cauchy distribution, the Levy distribution, the q-exponential distribution, the q-Gaussian distribution, the normal distribution, the Student's t-distribution, and the Laplace distribution.