The switching speed of light polarization plays a crucial role in determining the upper‐limit bandwidth of applications like optical communications and laser microscopy. However, conventional polarization elements based on macroscopic anisotropic crystals like birefringent crystals and chalcogenide glasses are either static or restricted by the low switching time of about hundreds of picoseconds. Here, a femtosecond‐scale all‐optical polarization controlling method is proposed through engineering the excited hot electrons in a graphene‐loaded metasurface. Remarkably, a giant polarization orientation from left‐handed polarization (LCP) to right‐handed polarization (RCP) in the Poincaré sphere (≈80° rotation) is realized within only 200 fs in the mid‐infrared. With pumping, the dedicated polarization‐sensitive design allows the metasurface to exhibit a consistent resonance blueshift as the transient increase of the hot‐electron temperature in graphene for y‐polarized incidence. This polarization conversion approach features a giant modulation range and enables the reflected light to be dynamically and arbitrarily modulated into RCP, LCP, and linear states at femtosecond timescale. A few logic operations “AND”, “OR”, “NAND”, and “XOR” based on this method are also demonstrated by monitoring the normalized Stokes parameters. It is believed that this work may find practical application in next‐generation signal‐processing systems with large capacity.