A study of the zero field and field induced magnetic states of the frustrated rare earth tetraboride HoB 4 has been carried out using single crystal neutron diffraction complemented by magnetization measurements. In zero field, HoB 4 shows magnetic phase transitions at T N1 = 7.1 K to an incommensurate state with a propagation vector (δ,δ,δ ), where δ = 0.02 and δ = 0.43 and at T N2 = 5.7 K to a noncollinear commensurate antiferromagnetic structure. Polarized neutron diffraction measurements in zero field have revealed that the incommensurate reflections, albeit much reduced in intensity, persist down to 1.5 K despite antiferromagnetic ordering at 5.7 K. At lower temperatures, application of a magnetic field along the c axis initially re-establishes the incommensurate phase as the dominant magnetic state in a narrow field range, just prior to HoB 4 ordering with an up-up-down ferrimagnetic structure characterized by the (h k 1 3 )-type reflections between 18 and 24 kOe. This field range is marked by the previously reported M/M sat = 1 3 magnetization plateau, which we also see in our magnetization measurements. The region between 21 and 33 kOe is characterized by the increase in the intensity of the antiferromagnetic reflections, such as (100), the maximum of which coincides with the appearance of the narrow magnetization plateau with M/M sat ≈ 3 5 . Further increase of the magnetic field results in the stabilization of a polarized state above 33 kOe, while the incommensurate reflections are clearly present in all fields up to 59 kOe. We propose the H -T phase diagram of HoB 4 for the H c containing both stationary and transitionary magnetic phases which overlap and show significant history dependence.