We analyze the effect of the Pauli-Fierz mass term on a recently established, new massive gravity theory in three space-time dimensions. We show that the Pauli-Fierz mass term makes the new massive gravity theory non-unitary. Moreover, although we add the gravitational Chern-Simons term to this model, the situation remains unchanged and the theory stays non-unitary in spite that structure of the graviton propagator is greatly changed. Thus, the Pauli-Fierz mass term is not allowed to coexist with massgenerating higher-derivative terms in the new massive gravity.