The polynomial functions of an algebra preserve all congruence relations. In addition, if the algebra is finite, they preserve the labelling of the congruence lattice in the sense of Tame Congruence Theory. The question is for which algebras every congruence preserving function, or at least every function that preserves the labelling of the congruence lattice, is a polynomial function. In this paper, we investigate this question for finite algebras that have a group reduct.