The construction sector, in modern times, is faced by a myriad of challenges primarily due to the increase in the urban population and dwindling natural resources that facilitate the production of construction materials. Furthermore, higher awareness on climate change is forcing companies to rethink their strategies in developing more sustainable construction materials. Diverse types of agro-waste ranging from rice husk ash (RHA), sugarcane bagasse ash (SCBA), and bamboo leaves ash (BLA) among others have been identified as potent solutions in the development of sustainable construction materials. In this review paper, six different construction materials, made using agro-waste products, are examined. The materials include brick/masonry elements, green concrete, insulation materials for buildings, reinforcement materials for buildings, particleboards, and bio-based plastics. The main criterion adopted in selecting the materials regards their popularity and wide-scale use in modern construction applications. Additionally, as this research emphasizes identifying alternative approaches to develop sustainable construction materials, the focus is directed toward mainstream materials whose continued use has an adverse impact on the environment. The findings obtained from the review showed that the use of agro-waste to develop sustainable construction materials was effective, as the developed materials adhered to established building standards. Therefore, this indicates that agro-waste materials have the potential to replace conventional construction materials and hence achieve economic, environmental, and social sustainability in the long run.