Lateral driving behavior analysis is the foundation of freeway cross-section design and the focus of road safety research. However, the factors that influence vehicle lateral driving behavior have not been clearly explained. The dataset of the natural driving trajectory of freeways is used in this study to analyze vehicle lateral driving behavior and trajectory characteristics. As vehicle trajectory characteristic indicators, parameters such as preferred trajectory deviation and standard deviation are extracted. The effects of lane position, speed, road safety facilities, and vehicle types on freeway trajectory behavior are investigated. The results show that lane width and lane position significantly impact vehicle trajectory distribution. As driving speed increases, the lateral distance between vehicles in the inner lane and the guardrail tends to increase. In contrast, vehicles in the outside lane will stay away from the road edge line, and vehicles in the middle lane will stay away from the right lane dividing line when the speed increases. Statistical analysis shows that the preferred trajectory distribution of the same vehicle type in different lane positions is significantly different among groups (Cohen’s d > 0.7). In the same lane, the lateral position characteristics of the center of mass of different vehicle types are basically the same (Cohen’s d < 0.35). This work aims to explain what variables cause trajectory deviation behaviors and how to design traffic safety facilities (guardrail and shoulder) and lane width to accommodate various vehicle types and design speeds.