The use of reclaimed asphalt pavement (RAP) is necessary for sustainable and cost-effective road infrastructure construction. This research investigates the effect of the area of weak zones (WZ) on the resilient modulus (MRT) of mixtures of fresh asphalt with 20% RAP. Experimentation on fresh asphalt–RAP mixtures comprising Superpave (SP-A, SP-B) and Asphalt Institute (MS-2) gradations with 20/30, 40/50, 60/70 and 80/100 penetration grade binders was carried out. WZ were determined based on the analysis of magnified digital images of asphalt specimens obtained using optical microscopy. This study demonstrates that the 20/30 grade binder caused an increase in the MRT at 25 °C up to 1.8, 2.9 and 9.2 times for a 0.1 s load duration, and 2.4, 3.0 and 9.7 times for a 0.3 s load duration. In contrast, improvement at 40 °C was observed to be up to 1.9, 3.1 and 9.7 times for a 0.1 s load duration, and 1.9, 3.0 and 12.4 times for a 0.3 s load duration in comparison with 40/50, 60/70 and 80/100 grade binders, respectively. Experimental data were validated by factorial analysis. Power trendline equations were also developed between MRT and WZ to explain the effect of gravel particle orientation on the sustainable resilience of surface course mixtures.