Field-Programmable Gate Arrays (FPGAs) are versatile, reconfigurable integrated circuits that can be used as hardware accelerators to process highly-sensitive data. Leaking this data and associated cryptographic keys, however, can undermine a system's security. To prevent potentially unintentional interactions that could break separation of privilege between different data center tenants, FPGAs in cloud environments are currently dedicated on a per-user basis. Nevertheless, while the FPGAs themselves are not shared among different users, other parts of the data center infrastructure are. This paper specifically shows for the first time that powering FPGAs, CPUs, and GPUs through the same power supply unit (PSU) can be exploited in FPGA-to-FPGA, CPU-to-FPGA, and GPU-to-FPGA covert channels between independent boards. These covert channels can operate remotely, without the need for physical access to, or modifications of, the boards. To demonstrate the attacks, this paper uses a novel combination of "sensing" and "stressing" ring oscillators as receivers on the sink FPGA. Further, ring oscillators are used as transmitters on the source FPGA. The transmitting and receiving circuits are used to determine the presence of the leakage on off-the-shelf Xilinx boards containing Artix 7 and Kintex 7 FPGA chips. Experiments are conducted with PSUs by two vendors, as well as CPUs and GPUs of different generations. Moreover, different sizes and types of ring oscillators are also tested. In addition, this work discusses potential countermeasures to mitigate the impact of the cross-board leakage. The results of this paper highlight the dangers of shared power supply units in local and cloud FPGAs, and therefore a fundamental need to rethink FPGA security for shared infrastructures.