The successive magnetic phase transitions of HoB 4 with T N1 ¼ 7:1 K and T N2 ¼ 5:7 K have been investigated in detail by neutron and X-ray diffraction. Since Ho ions in the c-plane form a Shastry-Sutherland lattice, geometrical frustration has been expected in this compound. Below T N1 , HoB 4 exhibits an incommensurate antiferromagnetic order with q ¼ ð; ; 0 Þ ( ¼ 0:022, 0 ¼ 0:43), followed by a first-order transition at T N2 into a commensurate order with q ¼ ð0; 0; 0Þ, where four magnetic moments are antiferromagnetically coupled in a unit cell. After determining the magnetic structures of the two phases by neutron powder diffraction, we pay particular attention to diffuse scatterings responsible for the critical behavior. Very broad diffuse scatterings, extending almost to the zone boundary, are observed already in the paramagnetic phase at around q ¼ ð0; 0; 0Þ and ð0; 0; 1=2Þ with magnetic correlation lengths of < 10 Å . Below T N1 , a strong diffuse scattering develops in a more concentrated region at around q ¼ ð0; 0; 0 Þ with $ 30 Å , followed by the appearance of a Bragg peak at ð; ; 0 Þ. Simultaneously, a sharp Bragg peak corresponding to q ¼ ð0; 0; 0Þ also develops at ð1; 0; 1Þ upon diffuse scattering. This complex critical behavior is discussed from the viewpoint of competing order parameters of magnetic and quadrupolar origins. The striking resemblance of the critical behavior to that of HoB 2 C 2 with a square lattice of Ho is interpreted as a manifestation of a universality for a threedimensional system with competing magnetic and quadrupolar order parameters and a long-ranged RKKY interaction. It is considered that the microscopic difference in interionic interaction due to the difference in crystal structure becomes less prominent in the critical behavior as the correlation length becomes much larger than the lattice constant. On the other hand, the strong diffuse scattering in HoB 4 , which is hardly detected in HoB 2 C 2 , might be ascribed to geometrical frustration. The softening of the elastic constant in HoB 4 and HoB 2 C 2 is also discussed.