$CP$ violation may play an important role in Baryogenesis in early universe, and should be examined at colliders comprehensively. We study $CP$ properties of $HVV$ vertexes between Higgs and gauge boson pairs with defining a $CP$ violation phase angle $\xi$, which indicates the mixture of $CP$-even and $CP$-odd Higgs states in $HVV$ in new physics. A series of $HVV$ amplitudes $H\to\gamma\gamma, H\to\gamma V\to \gamma \ell\ell$, and $H\to VV\to 4\ell$ with $CP$ phase angle are studied systematically, which explains explicitly why $CP$ violation could only be probed in $4\ell$ process independently. We get a novel amplitude decomposition relation which illustrates if two preconditions (multilinear momentum dependent vertexes and current $J_\mu$ of $V\to \ell^+ \ell^-$ is vector-like) are satisfied, an high-point amplitude can be decomposed into a summation of a series of low-point amplitudes. As a practical example, the amplitude of $H\to\gamma V\to \gamma \ell\ell$, and $H\to VV\to 4\ell$ processes can be decomposed into summation of many $H\to\gamma\gamma$ amplitudes. Meanwhile, we calculate these amplitudes in the framework of on-shell scattering amplitude method, with considering both massless and massive vector gauge bosons with $CP$ violation phase angle. The above two approaches provides consistent results and exhibit clearly the $CP$ violation $\xi$ dependence in the amplitudes. Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. Article funded by SCOAP3 and published under licence by Chinese Physical Society and the Institute of High Energy Physics of the Chinese Academy of Science and the Institute of Modern Physics of the Chinese Academy of Sciences and IOP Publishing Ltd.