A three-dimensional combined finite-discrete element element method (FDEM), parallelized by a general-purpose graphic-processing-unit (GPGPU), was applied to identify the fracture process of rough concrete–rock joints under direct shearing. The development process of shear resistance under the complex interaction between the rough concrete–rock joint surfaces, i.e., asperity dilatation, sliding, and degradation, was numerically simulated in terms of various asperity roughness under constant normal confinement. It was found that joint roughness significantly affects the development of overall joint shear resistance. The main mechanism for the joint shear resistance was identified as asperity sliding in the case of smoother joint roughness and asperity degradation in the case of rougher joint asperity. Moreover, it was established that the bulk internal friction angle increased with asperity angle increments in the Mohr–Coulomb criterion, and these results follow Patton’s theoretical model. Finally, the friction coefficient in FDEM appears to be an important parameter for simulating the direct shear test because the friction coefficient affects the bulk shear strength as well as the bulk internal friction angle. In addition, the friction coefficient of the rock–concrete joints contributes to the variation of the internal friction angle at the smooth joint than the rough joint.