Exchange bias is usually observed in systems with an interface between a ferromagnet and an antiferromagnet. As a result of the competition between antiferromagnetic and ferromagnetic interactions at the interface, this effect may involve aspects of glassy behavior. However, the relationship between spin glass and exchange bias is still under investigation. The formation origin of magnetic anisotropy in spin glasses to reveal their relation is a key research issue. Here, we comparatively study the magnetic and transport exchange bias in a spin glass CuMn after zero-field cooling from an unmagnetized state. We show that the magnetic anisotropy causing the zero-field-cooled exchange bias can be resolved by a linear superposition of unidirectional and uniaxial parts through transport measurements. The former provides a source of net magnetic moments, whereas the latter provides a mechanism for its pinning, which induces the exchange anisotropy for the zero-field-cooled exchange bias during a virgin magnetization process.