Under the strategies of low-carbon and environmental protection, promoting green technology innovation to achieve carbon neutrality in the construction field has become a universal goal. As the building material with the highest consumption, concrete has gradually begun to transform into a multi-functional and intelligent product. Therefore, the research on carbon fiber-reinforced cement-based composites (CFRCs) is of relative interest. It mainly uses carbon fibers (CFs) with high elasticity, strength, and conductivity to disperse evenly into the concrete as a functional filler, to achieve the intelligent integration of concrete structures and function innovatively. Furthermore, the electrical conductivity of CFRC is not only related to the content of CFs and environmental factors but also largely depends on the uniform dispersion and the interfacial bonding strength of CFs in cement paste. This work systematically presents a review of the current research status of the enhancement and modification mechanism of CFRC and the evaluation methods of CF dispersion. Moreover, it further discusses the improvement effects of different strengthening mechanisms on the mechanical properties, durability, and smart properties (thermoelectric effect, electrothermal effect, strain-sensitive effect) of CFRC, as well as the application feasibility of CFRC in structural real-time health monitoring, thermal energy harvesting, intelligent deformation adjustment, and other fields. Furthermore, this paper summarizes the problems and challenges faced in the efficient and large-scale applications of CFRCs in civil engineering structures, and accordingly promotes some proposals for future research.