An as-grown domain structure in nominally pure and Mn-doped calcium orthovanadate (CVO) crystals was studied by several methods of domain imaging: optical microscopy, piezoelectric force microscopy, and Cherenkov-type second harmonic generation. The combination of imaging methods provided an opportunity for comprehensive study of the domain structure on the polar surface and in the bulk of the samples. It was shown that, in nominally pure CVO crystals, an irregular 3D maze of rounded domains, with charged walls, essentially tilted from the polar direction, was present. It was proposed that the domain structure was formed just below the phase transition temperature and persisted during subsequent cooling. Such behavior is due to effective bulk screening of the depolarization field and a low value of the pyroelectric field which appears during cooling. The revealed formation of triangular domains and flat fragments of domain walls in Mn-doped CVO was attributed to polarization reversal under the action of the polar component of the pyroelectric field, above the threshold value for domain switching. This fact represents the first observation of the domain switching in CVO crystals.