Recycling aluminosilicate-based solid wastes is imperative to realize the sustainable development of constructions. By using alkali activation technology, aluminosilicate-based solid wastes, such as furnace slag, fly ash, red mud, and most of the bio-ashes, can be turned into alternative binder materials to Portland cement to reduce the carbon footprint of the construction and maintenance activities of concrete structures. In this paper, the chemistry involved in the formation of alkali-activated materials (AAMs) and the influential factors of their properties are briefly reviewed. The commonly used methods, including X-ray diffraction (XRD), scanning electron microscopy (SEM), thermogravimetric analysis (TG), nuclear magnetic resonance spectroscopy (NMR), and X-ray pair distribution function technology, to characterize the microstructure of AAMs are introduced. Typical characterization results of AAMs are shown and the limitations of each method are discussed. The main challenges, such as shrinkage, creep, efflorescence, carbonation, alkali–silica reaction, and chloride ingress, to conquer for a wider application of AAMs are reviewed. It is shown that several performances of AAMs under certain circumstances seem to be less satisfactory than traditional portland cement systems. Existing strategies to improve these performances are reviewed, and recommendations for future studies are given.