The current platoon control strategies of connected autonomous vehicles (CAVs) focus on controlling the fixed intervehicle distance, i.e., the string stability of the platoon system. Here, we aimed to design a CAV platoon control strategy based on a constraint-following approach to solve the problem of platoon starting. As the resistance of the vehicle during driving varies with time, this study regarded the CAV platoon system as a changing dynamic system and introduced the Udwadia–Kalaba (U–K) approach to simplify the solution. Apart from adding an equality constraint, unlike most other studies, this study imposed a bilateral inequality constraint on the intervehicle distance between successive CAVs to prevent collisions. Meanwhile, a diffeomorphism method was introduced to transform the bounded state into an unbounded state. The proposed control strategy could render each CAV compliant with both the original imposed bilateral inequality constraint and the equality constraint. The former avoids collisions, and the latter indicates the string stability of the designed CAV platoon system. The effectiveness of the proposed controller was verified by numerical experiments. The gap errors tend to converge to zero, which is not amplified by the propagation of traffic flow.