In this study, a transversal ultrasonic vibration-assisted riveting (TUVAR) process was developed to improve the uniformity of CFRP riveted lap joint interference-fit size, which provided a possibility for the quantization of riveted joint interference-fit sizes. The relationship between the process parameters of vibration amplitude, vibration duration, and roughness with interference-fit sizes by algorithms, through the minimum coefficient variance of interference-fit size (ICV-min) to confirm the riveting process parameters of the quantized average interference-fit sizes (IA). The experimental verification results showed that the mean absolute percentage error of measured IA and predicted IA is less than 10%. Furthermore, the tensile tests were carried out to investigate the effect of interference-fit size {1.4%, 1.6%, 1.8%, and 2.0%} on mechanical performances of CFRP riveted lap joints by TUVAR, and the tensile strength presents first-up then down with the interference-fit size increase, the maximum ultimate tensile strength is the riveted lap joint with the interference-fit size of 2.0%. Hence, the quantitative optimization method can well predict the riveting process parameters corresponding to the most uniform interference-fit size.