The negative environmental impacts of Portland cement as a binder in the construction industry have created a growing impetus to develop sustainable alternative binders. Various types of clay have been considered as potential cement replacements. The impact of clays as cement replacement depends on the dosage and treatment methods. This paper presents a comprehensive review to determine the effects of different types of clay on the fresh and hardened properties of concrete mixtures by analyzing the experimental database reported by the literature, including raw, calcined, modified, nano, and organo. This study intends to show the process of optimizing the use of clay in concrete, the reason behind converting raw clay to modified types, and research gaps through a comparison study between different types of clays. The present review study shows that clay-based concrete mixtures have higher thixotropy and yield stress values, improving shape stability. This results in lower early-age shrinkage of the concrete. However, the high floc strength of clay-based concrete causes a reduction in flowability. Treatment methods of raw clay, such as calcination and nano-sized clay particles, improve concrete compressive strength. General results of the previous studies highlight that all types of clay investigated positively affect the resistance of concrete to environmental attack.