Riveting is the most important way to connect metal sheets, which is widely used in the connection of aircraft components. In this paper, the effect of different hole diameters on the shear properties of riveted lap joints were studied from the perspective of practical application. Considering the symmetry and the calculation time of the model, a 2D axisymmetric finite element method is established with the help of ABAQUS commercial finite element software, the validity of the finite element model is verified by experiment tests. Because the interference distribution has an important influence on the mechanical properties of riveted lap joints, the interference distribution and material flow characteristics in riveting process are analyzed in detail by using finite element method, and the shear characteristics of riveted lap joints in tensile process are explained. The variation of hole diameter with shear force under different squeeze force was obtained by shear test in order to explain the effect of hole diameter on the shear mechanical properties of riveted lap joint. In addition, the fracture mode and microstructure of the rivet shank were characterized by SEM and the formation process of brittleness and plastic fracture is discussed. Finally, the shear failure mechanism of riveted lap joint is analyzed in detail to provide guidance for engineering application. The test results show that all the specimens are both brittle and plastic mixed fracture modes of rivet shank, and the shear strength of the rivet increases with the increase of the hole diameter. Compared with increasing the squeeze force, increasing the hole diameter can effectively improve the shear strength of the riveted lap joint.