The joints are existing throughout the underground rock mass. It is of great significance to investigate the shear performance of the rock mass to maintain the stability of the underground structure. In this study, we conducted orthogonal tests to determine the proportion of rock-like materials, and used JRC curves to make specimen molds and then prepare the specimens. We conducted straight shear tests and uniaxial compression tests to determine the various mechanical parameters of the rock-like materials. Next, we carried out the compression and shear tests to investigate the shear characteristics of the specimens, and study the damage pattern and shear strength of the jointed rock mass under different confining pressures and roughness levels. The mesoscopic displacements in the shear process of joints were analyzed by using ABAQUS. The test results show that the effect of the confining pressure on the shear strength of the joint plane is relatively obvious, and a larger confining pressure indicates a larger shear strength. The effects of different joint plane roughness and shear rated on the shear characteristics of the joint plane are also significant. The mesoscopic displacement difference inside the joint plane with higher roughness is relatively large, and the stress concentration phenomenon is obvious and lasts longer, which leads to the faster destruction of the specimen with higher roughness and the higher destruction degree. Therefore, we suggest that the priority should be given to the reinforcement of jointed rock mass with high roughness during the construction to prevent sudden destabilization and failure.