Metasurfaces provide a compact and powerful platform for manipulating the fundamental properties of light, and have shown unprecedented capabilities in both optical holographic display and information encryption. For increasing information display/storage capacity, metasurfaces with more polarization manipulation channel and full‐color holographic functionality are now an urgent requirement. Here, a minimalist dielectric metasurface with the capability of full‐color holography encoded with arbitrary polarization is proposed and experimentally demonstrated. Without the daunting exploratory and computational problem in nanostructure searching, full‐color holographic images can be multiplexed into arbitrary polarization channels through vectorial ptychography and k‐space ptychography based on tetratomic macropixel geometric phase metasurfaces. Thanks to the full degree of freedom tuning in polarization and color spaces, the application scenarios such as holographic 3D imaging and information encryption are realized. The strategy exhibits promising potential in applications of 3Dl display, augmented/virtual reality, high‐density data storage, and encryption.