In this paper, synthesis and characterization of a novel CeO2/nanoclay nanocomposite (NC) and its effects on IFT reduction and wettability alteration is reported in the literature for the first time. The NC was characterized using scanning electron microscopy (SEM), X-ray diffraction (XRD), Fourier-transform infrared spectroscopy (FTIR), thermogravimetric analysis (TGA), energy-dispersive X-ray spectroscopy (EDS), and EDS MAP. The surface morphology, crystalline phases, and functional groups of the novel NC were investigated. Nanofluids with different concentrations of 100, 250, 500, 1000, 1500, and 2000 ppm were prepared and used as dispersants in porous media. The stability, pH, conductivity, IFT, and wettability alternation characteristics of the prepared nanofluids were examined to find out the optimum concentration for the selected carbonate and sandstone reservoir rocks. Conductivity and zeta potential measurements showed that a nanofluid with concentration of 500 ppm can reduce the IFT from 35 mN/m to 17 mN/m (48.5% reduction) and alter the contact angle of the tested carbonate and sandstone reservoir rock samples from 139° to 53° (38% improvement in wettability alteration) and 123° to 90° (27% improvement in wettability alteration), respectively. A cubic fluorite structure was identified for CeO2 using the standard XRD data. FESEM revealed that the surface morphology of the NC has a layer sheet morphology of CeO2/SiO2 nanocomposite and the particle sizes are approximately 20 to 26 nm. TGA analysis results shows that the novel NC has a high stability at 90 °C which is a typical upper bound temperature in petroleum reservoirs. Zeta potential peaks at concentration of 500 ppm which is a sign of stabilty of the nanofluid. The results of this study can be used in design of optimum yet effective EOR schemes for both carbobate and sandstone petroleum reservoirs.