Computational approaches could provide a viable and cost-effective alternative to expensive experiments for accurately evaluating the nonlinear constitutive behavior of cementitious nanocomposite materials. In the present study, the mechanical properties of cement paste reinforced with multi-wall carbon nanotubes (MWCNTs) are examined experimentally and numerically. A multiscale computational approach is adopted in order to verify the experimental results. For this scope, a random sequential adsorption algorithm was developed to generate non-overlapping matrix-inclusion three-dimensional (3D) representative volume elements (RVEs), considering the inclusions as straight elements. Nonlinear finite element analyses (FEA) were performed, and the homogenized elastic and inelastic mechanical properties were computed. The use of a multiscale computational approach to accurately evaluate the nonlinear constitutive behavior of cementitious materials has rarely been explored before. For this purpose, the RVEs were analyzed both in pure tension and compression. Young’s modulus as well compressive and tensile strength results were compared and eventually matched the experimental values. Moreover, the effect of MWCNTs on the nonlinear stress–strain behavior of reinforced cement paste was noted. Subsequently, three-point bending tests were conducted, and the stress–strain behavior was verified with FEA in the macro scale. The numerical modeling reveals a positive correlation between the concentration of MWCNTs and improved mechanical properties, assuming ideal dispersion. However, it also highlights the impact of practical limitations, such as imperfect dispersion and potential defects, which can deteriorate the mechanical properties that are observed in the experimental results. Among the different cases studied, that with a 0.1 wt% MWCNTs/CP composite demonstrated the closest agreement between the numerical model and the experimental measurements. The numerical model achieved the best accuracy in estimating the Young’s modulus (underestimation of 13%), compressive strength (overestimation of 1%), and tensile strength (underestimation of 6%) compared to other cases. Overall, these numerical findings contribute significantly to understanding the mechanical behavior of the nanocomposite material and offer valuable guidance for optimizing cement-based composites for engineering applications.