We have switched polarization in calcium orthovanadate single crystal with as-grown domain structure consisting of isolated domains with charged domain walls (CDWs) located in the bulk using pretreatment by ac field and subsequent switching in dc field at the elevated temperature. The formation of the domain ledges at the CDW in the bulk and their growth in the polar direction has been revealed. The isolated domains with optically well-defined walls appeared when the ledge tops reached the surface, their shape and sizes remaining constant during further switching. Unlike usual continuous domain wall motion, we have observed the discrete switching by arising of the isolated domains without any input of the traditional domain nucleation at the polar surface. The obtained results have been explained under the assumption that at the used experimental conditions, the applied field is above the threshold value for ledge nucleation at CDW in the bulk, but below the threshold for domain wall motion at the surface. Thus, we have obtained the discrete switching by ledge growth without any sideways motion of the domain walls and domain coalescence. The nonuniform evolution of the domain structure at the surface is due to the dependence of the switching rate on the distance from CDW to the polar surface, which is random in the studied domain structure.