Human-vehicle collaborative driving (HVCD) has a novel concept of an intermediate stage from the development of a normal smart vehicle to self-driving vehicle, requires the vehicle's decisionmaking authority equivalent to the human driver's operation. As the self-driving vehicle cannot achieve full-condition driving without human driver yet and still have a long time to go, the collaborative driving can exploit both machine and human specialties, which requires human and vehicle intelligence to control the vehicle simultaneously and work together. However, there were a few studies focusing on the HVCD that experimental environment and evaluation methods are rarely discussed, even under the simulation conditions, which limits the development of collaborative driving. To bridge the significant gaps, this paper designed simple semi-physical simulation system architecture for the HVCD. First, the simple designed system can achieve basic functions, including human control input as the normal driver, vehicle intelligence with local and global sensor data process and decision fusion. The presented system can use and process this multisource information to achieve the HVCD in the simulated environment. Second, the evaluation method based on vehicle driving safety, stability, and rapidity is proposed and discussed, which matters most for system performance test and feasibility verification. In addition, experiments with different decision fusion conditions were presented and the experimental results show that the proposed simple semi-physical HVCD simulation system can work smoothly and reflect the practical scenario. Furthermore, the performance of different fusion weights experiments shows the advantages of both human driver and vehicle intelligence, which provides a reference for the HVCD related research improvement. INDEX TERMS Human-vehicle collaborative driving, simulation system design, vehicle driving performance evaluation.