The p-GaN gated E-mode HEMTs (P-HEMTs) are being extensively studied for emerging power electronics. However, the devices still suffer from performance trade-off among threshold voltage (VTH), output drain current (IDS), and breakdown voltage (VBD). Herein, we propose P-HEMTs with a stair-like p-GaN cap layer to boost their performance. The p-GaN cap layer is composed of four discrete p-GaN staircases with the decreased thickness to the right (Right-Stair P-HEMT, RSP-HEMT) or to the left (Left-Stair P-HEMT, LSP-HEMT). It is found that the RSP-HEMT simultaneously achieves the 1.3-times increased IDS, 160 V enhanced VBD, and improved VTH stability against drain-induced barrier lowering (DIBL) effects, compared with the LSP-HEMT and the conventional BaSeline P-HEMT (BSP-HEMT). These device merits should be attributed to the effective manipulation of the lateral electric field (EF) under all bias conditions by the unique band structures enabled by the RSP configuration. Such EF manipulation strategies to improve device characteristics offer us helpful guidance and insights to further propel the development of high-performance E-mode P-HEMTs of the future.