In order to improve the running quality of trains on a ballastless track, the influence of the CRTS I ballastless track with different structures (flat-type and frame-type tracks) is investigated with respect to the aerodynamic characteristics of high-speed trains. In the present paper, the aerodynamic force changes on the head, middle, tail, and whole car of the high-speed train were studied under two conditions, with crosswind and without crosswind, and the influence of different crosswind speeds (10, 15, 20, 25, 30 m/s) on the aerodynamic force of the train was analyzed. The pressure and flow field distribution characteristics were also studied, and the reasons for the different aerodynamic characteristics of different track structures and trains running in different wind environments were analyzed, respectively. The results indicate that the ballastless track structure obviously influences the aerodynamic characteristics of the high-speed train. When there is no natural wind, compared with the flat track, the frame track reduces the drag and lateral forces of the train but increases the lift force. The frame track causes the drag force of the whole vehicle to decrease slightly (the maximum ratio is 2.15%), the lift force increases significantly (the maximum ratio is 12.55%), and the lateral force obviously decreases (the maximum ratio is 52.43%). The lift and lateral forces of the middle car are most affected, which is because the frame structure changes the vortex motion state of the middle car. Compared with the flat track, the drag force of each car on the frame track is reduced under the crosswind; the lift force of each car is increased, and the maximum increase in the lift force of the head, middle, and tail cars is 5.60%, 2.55%, and 3.63%, respectively; the lateral force of the tail car increases greatly at a wind speed of 15 m/s, reaching 6.84%. Due to the existence of the frame structure, the space under the vehicle increases, resulting in a decrease in the airflow rate and an increase in local pressure, which leads to changes in the train’s aerodynamic force. Meanwhile, the train’s aerodynamic change under the crosswind is smaller than that when there is no wind.