On‐chip mode exchangers play very important roles for multimode photonics, which has drawn much attention for many applications. However, it is still very challenging to achieve compact mode exchangers with high performances in an ultra‐broad bandwidth. Here a universal approach of on‐chip mode manipulations is proposed by manipulating the multimode excitations/interferences, and ultra‐compact mode exchangers are realized with low excess losses (ELs) and high extinction ratios (ERs) using special metamaterial structures. The presented TEi–j mode exchangers enable the exchange between the TEi and TEj modes in an ultra‐broad bandwidth. The designed TE0–1 and TE0–2 mode exchangers have footprints of 1.3 × 2.7 µm2 and 1.9 × 2.9 µm2, respectively. Theoretically, they have ELs ≈ 0.2 dB and ERs = 19–32 dB at 1550 nm, while their bandwidths for achieving ER > 10 dB reach 340–400 nm. The fabricated devices have measured ELs of 0.2–0.3 dB and ERs above 12.0–13.2 dB in the setup‐limited wavelength range of 1520–1607 nm. The proposed approach is flexibly extendable to enable on‐chip mode exchanges, which are promising for multimode silicon photonics in many applications.