Vertically (off-plane) ferroelectric ordering in ultra-thin films has been pursued for decades. We predict the existence of intrinsic vertical polarization orderings in ultra-thin metal chalcogendiphosphates (MCDs). Taking CuInP2Se6 as an example, the first-principles calculation and electrostatic-energy model show that, under the open-circuit boundary condition, the ground state of bulk CuInP2Se6 is ferroelectric (FE) while that of monolayer is antiferroelectric (AFE), and the critical thickness for this FE/AFE transition is around 6 layers. Interestingly, under the closedcircuit boundary condition, the FE state can hold even for monolayer. Particularly, because of the small energy difference but the large barrier between FE and AFE orderings, the FE state can be stabilized in free-standing monolayer, giving rise to intrinsic, off-plane two-dimensional ferroelectrics. Applying Monte Carlo simulations, we further calculate the ferroelectric Curie temperature (Tc) and electric hysteresis.