We study the characteristic polynomial of random permutation matrices following some measures which are invariant by conjugation, including Ewens' measures which are one-parameter deformations of the uniform distribution on the permutation group. We also look at some modifications of permutation matrices where the entries equal to one are replaced by i.i.d uniform variables on the unit circle. Once appropriately normalized and scaled, we show that the characteristic polynomial converges in distribution on every compact subset of C to an explicit limiting entire function, when the size of the matrices goes to infinity. Our findings can be related to results by Chhaibi, Najnudel and Nikeghbali on the limiting characteristic polynomial of the Circular Unitary Ensemble [4].