We present a review of gravitating particle-like and black hole solutions with non-Abelian gauge fields. The emphasis is given to the description of the structure of the solutions and to the connection with the results of flat space soliton physics. We describe the Bartnik-McKinnon solitons and the non-Abelian black holes arising in the Einstein-Yang-Mills theory, and consider their various generalizations. These include axially symmetric and slowly rotating configurations, solutions with higher gauge groups, Λ-term, dilaton, and higher curvature corrections. The stability issue is discussed as well. We also describe the gravitating generalizations for flat space monopoles, sphalerons, and Skyrmions.