Highly constrained trajectory optimization problems are usually difficult to solve. Due to some real-world requirements, a typical trajectory optimization model may need to be formulated containing several objectives. Because of the discontinuity or nonlinearity in the vehicle dynamics and mission objectives, it is challenging to generate a compromised trajectory that can satisfy constraints and optimize objectives. To address the multi-objective trajectory planning problem, this study applies a specific multiple-shooting discretization technique with the newest NSGA-III optimization algorithm and constructs a new evolutionary optimal control solver. In addition, three constraint handling algorithms are incorporated in this evolutionary optimal control framework. The performance of using different constraint handling strategies is detailed and analyzed. The proposed approach is compared with other well-developed multiobjective techniques. Experimental studies demonstrate that the present method can outperform other evolutionary-based solvers investigated in this paper with respect to convergence ability and distribution of the Pareto-optimal solutions. Therefore, the present evolutionary optimal control solver is more attractive and can offer an alternative for optimizing multi-objective continuous-time trajectory optimization problems.