The present work explores the optimal flight of aero-assisted reentry vehicles during the atmospheric entry flight phase with the consideration of both deterministic and control chance constraints. To describe the mission profile, a chance-constrained optimal control model is established. Due to the existence of probabilistic constraints (chance constraints), standard numerical trajectory planning algorithms cannot be directly applied to address the considered problem. Hence, we firstly present an approximation-based strategy to replace the probabilistic constraint by a deterministic version. In this way, the transformed optimal control model becomes solvable for standard trajectory optimization methods. In order to obtain enhanced computational performance, an alternative convex-relaxed optimal control formulation is also given. This is achieved by convexifying the vehicle nonlinear dynamics/constraints and by introducing a convex probabilistic constraint handling strategy. Numerical simulations are provided to demonstrate the effectiveness of these two chance-constrained optimization approaches and the corresponding probabilistic constraint handling strategies.