To study the ride comfort of high-speed train passengers, a detailed human body–seat–vehicle–track coupled model is developed. This model adopts a dual-axis human-body dynamic model and considers the nonlinearity of the primary and secondary suspensions and the wheel–rail contact condition. Additionally, the input wheel/rail irregularity excitation data are measured for a high-speed railway line in China. According to the model, the root-mean-square (RMS) values of the acceleration for the passenger head at different positions of the car body are determined, and the effects of the car-body loading conditions, speed, and seat support parameters on the head vibration characteristics are studied. The results indicate that the RMS value is the highest for the passengers sitting in the front row near the window seat, followed by those sitting in the middle row of the car body. There are small differences in the RMS value between the passengers sitting on the left and right sides of the same row in the car body. At a certain speed, the track irregularity stimulates the rolling and pitching motions of the train. Therefore, effective methods should be implemented to avoid the inherent characteristics of the car body. Although an increase in the seat support stiffness intensifies the high-frequency vibration, it reduces the low-frequency vibration in the range of 0–15 Hz, reducing the RMS value of the head acceleration. When the train is loaded, the effect of passengers on the car body is similar to that of a damper or mass element, which can reduce the car-body vibration and enhance the ride comfort.