The rapid advancements in machine learning have exacerbated the interconnect bottleneck inherent in binary logic‐based computing architectures. An interesting approach to tackle this problem involves increasing the information density per interconnect, i.e., by switching from a two‐valued to a multi‐valued logic (MVL) architecture. However, current MVL implementations offer limited overall performance and face challenges in scaling to process data signals with radix (number of logic levels) even just above 3. In this work, a novel concept is introduced for implementation of a highly scalable and fully passive inverter based on the frequency‐domain phase‐only linear manipulation of the input MVL data signal, which is encoded in the amplitude variations of an electromagnetic wave along the time axis. As a key advantage, this solution is entirely independent of the input radix. The proposed design is implemented using an optical fibre Bragg grating device. Inversion of quaternary signals is experimentally demonstrated, as well as binary and ternary signals, at a remarkable operation speed of 32 GBaud, with an estimated energy consumption of 24 fJ/bit. The proposed method is universal and can be applied to any system that supports transmission and detection of coherent waves, such as microwave, plasmonic, mechanical, or quantum.