The squealer tip has significant influence on both the aerodynamic and heat transfer characteristics of the high-pressure turbine blade. However, due to the complexity of parameterization and meshing of the squealer and the complicated flow structure within the over-tip region, the existing squealer designs in the open literature have constant squealer heights. In this paper, the design space to the squealer height with non-uniform squealer height is extended and the new flow features it may bring are investigated. A parameterization system specifically designed for the non-uniform squealer height using five control parameters is implemented to automatically generate the geometry and hybrid meshes. Combining it with the multi-objective optimization system using genetic algorithms, a transonic turbine cascade squealer tip is optimized employing Reynolds-averaged Navier–Stokes k–ω shear stress transport model. The main objective of this study is to obtain a squealer configuration with the lowest total pressure loss coefficient and heat transfer coefficient. The optimum configuration with non-uniform squealer height achieves improvements in both the aerodynamic efficiency and the heat transfer performance, relative to the baseline conventional squealer tip geometry with the constant squealer height. Additionally, this work demonstrates that a flow structure in which the main flow forms a “blanket” below the leakage flow in the squealer is beneficial for aerothermal performance, especially reducing heat transfer losses, which provides valuable insight into the squealer tip design of advanced high-pressure turbines.