Shear strength is an essential index for the evaluation of soil stability. Test results of the shear strength of scaled coarse-grained soil (CGS for short) are usually not able to accurately reflect the actual properties and behaviors of in situ CGS due to the scale effect. Therefore, this study focuses on the influence of the scale effect on the shear strength of scaled CGS, which has an important theoretical significance and application for the strength estimation of CGS in high earth-rock dam engineering. According to previous studies, the main cause of the scale effect for scaled CGS is the variation of the gradation structure as well as the maximum particle size (dmax), in which the gradation structure as a characteristic parameter can be expressed by the gradation area (S). A total of 24 groups of test soil samples with different gradations were designed by changing the maximum particle size dmax and gradation area S. Direct shear tests were conducted in this study to quantitatively explore the effect of the gradation structure and the maximum particle size on the shear strength of CGS. Test results suggest that the shear strength indexes (i.e., the cohesion and internal friction angle) of CGS present an increasing trend with the improvement of the maximum particle size dmax, and thus a logarithmic function relationship among c, φ, and dmax is presented. Both cohesion (c) and internal friction angle (φ) are negatively related to the gradation area (S) in most cases. As a result, an empirical relationship between c, φ, and S is established based on the test results. Furthermore, a new prediction model of shear strength of CGS considering the scale effect is proposed, and the accuracy of this model is verified through the test results provided by relevant literature. Finally, the applicability of this model to different types of CGS is discussed.