Even though we have some proposals for the quantum theory of gravity like string theory or loop quantum gravity, we do not have any experimental evidence supporting any of these theories. Actually, we do not have empirical evidence pointing in the direction that we really need a quantum description of the gravitational field. In this scenario, several proposals for experimentally investigating quantum gravitational effects far from the Planck scale have recently appeared in literature, like gravitationally induced entanglement, for instance. An important issue of these approaches is the decoherence introduced by the quantum nature not only of the system under consideration but also from the gravitational field itself. Here, by means of the Feynman–Vernon influence functional, we study the decoherence of a quantum system induced by the quantized gravitational field—in the linearized gravity regime—and also by its own quantum nature. Our results may be significant in better understanding many phenomena like the decoherence induced by the gravitational time-dilation, the quantum reference frames, and the quantum equivalence principle.