Connected and automated vehicle (CAV) technologies have great potential to improve road safety. However, an emerging type of mixed traffic flow with human-driven vehicles (HDVs) and CAVs has also arisen in recent years. To improve the overall safety of this mixed traffic flow, a novel car-following model is proposed to control the driving behaviors of the above two types of vehicles in a platoon from the perspective of a mechanical system, mass-spring-damper (MSD) system. Furthermore, a quantitative index is proposed by incorporating the psychological field theory into the MSD model. The errors of spacing and speed in the car-following processes can be expressed as the accumulation of the virtual total energy, and the magnitude of the energy is used to reflect the danger level of vehicles in the mixed platoon. At the same time, the optimization model of minimum total energy is solved under the constraints of vehicle dynamics and the mechanical characteristics of the MSD system, and the optimal solutions are used as the parameters of the MSD car-following model. Finally, a mixed platoon composed of 3 CAVs and 2 HDVs without performing lane changing is tested using the driver-in-the-loop test platform. The test results show that, in the mixed platoon, CAVs can optimally adjust the intervehicle spacing by making full use of the braking distance, which also provides sufficient reaction time for the driver of HDV to avoid rear-end collisions. Furthermore, in the early stage of the emergency braking, the spacing error is the dominant factor influencing the car-following behaviors, but in the later stage of emergency braking, the speed error becomes the decisive factor of the car-following behaviors. These results indicate that the proposed car-following model and quantitative index are of great significance for improving the overall safety of the mixed traffic flow with CAVs and HDVs.