In this work, ultrasonic severe surface rolling (USSR), a new surface nanocrystallization technique, is used to prepare gradient nanostructure (GNS) on the commercial Q345 structural steel. The microstructure of the GNS surface layer is characterized by employing EBSD and TEM, and the result indicates that a nanoscale substructure is formed at the topmost surface layer. The substructures are composed of subgrains and dislocation cells and have an average size of 309.4 nm. The GNS surface layer after USSR processing for one pass has a thickness of approximately 300 μm. The uniaxial tensile measurement indicates that the yield strength of the USSR sample improves by 25.1% compared to the as-received sample with slightly decreased ductility. The nanoscale substructure, refined grains, high density of dislocations, and hetero-deformation-induced strengthening are identified as responsible for the enhanced strength. This study provides a feasible approach to improving the mechanical properties of structural steel for wide applications.