In this review, we discuss recent advances in the self-assembly of self-propelled colloidal particles and highlight some of the most exciting results in this field, with a specific focus on dry active matter. We explore this phenomenology through the lens of the complexity of the colloidal building blocks. We begin by considering the behavior of isotropic spherical particles. We then discuss the case of amphiphilic and dipolar Janus particles. Finally, we show how the geometry of the colloids and/or the directionality of their interactions can be used to control the physical properties of the assembled active aggregates, and we suggest possible strategies for how to exploit activity as a tunable driving force for self-assembly. The unique properties of active colloids lend promise to the design of the next generation of functional, environment-sensing microstructures able to perform specific tasks in an autonomous and targeted manner.