We have developed a method to predict the hardness of materials containing ultrastrong anionic polyhedra, dense atomic clusters, and layers stacked through van der Waals bonds on the basis of group electronegativity. By considering these polyhedra, clusters, and layers as groups that behave as rigid unities like superatoms bonding to other atoms or groups, the hardness values of materials such as oxysalts, T-carbon, and graphite were quantitatively calculated, and the results are consistent with the available experiments. We found that the hardness of materials containing these artificial groups is determined by the bonds between the groups and other atoms or groups, rather than by the weakest bonds. This work sheds light on the nature of materials hardness and the design of novel inorganic crystal materials.