Reconfigurable silicon photonic devices are widely used in numerous emerging fields such as optical interconnects, photonic neural networks, quantum computing, and microwave photonics. Currently, phase change materials (PCMs) have been extensively investigated as promising candidates for building switching units due to their strong refractive index modulation. Here, nonvolatile multilevel switching of silicon photonic devices with Ge2Sb2Te5 (GST) is demonstrated with In2O3 transparent microheaters that are compatible with diverse material platforms. With GST integrated on the silicon photonic waveguides and Mach‐Zehnder interferometers (MZIs), repeatable and reversible multilevel modulation of GST is achieved by electro‐thermally induced phase transitions. Particularly, the segmented switching unit of In2O3 and GST is proposed and demonstrated to be capable of producing about one order of magnitude larger temperature gradient than that of the nonsegmented unit, resulting in up to 64 distinguishable switching levels of 6‐bit precision, and fine‐tuning of the switching voltage pulses is promising to push the precision even further, to 7‐bit, or 128 distinguishable switching levels. The capability of precise multilevel phase‐change modulation is crucial to further facilitate the development of nonvolatile reconfigurable switches and variable attenuation devices as building blocks in large‐scale programmable optoelectronic systems.