This paper discusses a sustainable way to prepare construction materials from metallurgical slags. Steel slag, copper slag, lead-zinc slag, and electric furnace ferronickel slag are the most common metallurgical slags that could be used as supplementary cementitious materials (SCMs) and aggregates. However, they have some adverse effects that could significantly limit their applications when used in cement-based materials. The setting time is significantly delayed when steel slag is utilized as an SCM. With the addition of 30% steel slag, the initial setting time and final setting time are delayed by approximately 60% and 40%, respectively. Because the specific gravity of metallurgical slags is 10–40% higher than that of natural aggregates, metallurgical slags tend to promote segregation when utilized as aggregates. Furthermore, some metallurgical slags deteriorate the microstructure of hardened pastes, resulting in higher porosity, lower mechanical properties, and decreased durability. In terms of safety, there are issues with the soundness of steel slag, the alkali-silica reaction involving cement and electric furnace ferronickel slag, and the environmental safety concerns, due to the leaching of heavy metals from copper slag and lead-zinc slag.