Image noise is a common problem in light microscopy. This is particularly true in real-time live-cell imaging applications in which long-term cell viability necessitates low-light conditions. Modern denoisers are typically trained on a representative dataset, sometimes consisting of just unpaired noisy shots. However, when data are acquired in real time to track dynamic cellular processes, it is not always practical nor economical to generate these training sets. Recently, denoisers have emerged that allow us to denoise single images without a training set or knowledge about the underlying noise. But such methods are currently too slow to be integrated into imaging pipelines that require rapid, real-time hardware feedback. Here we present Noise2Fast, which can overcome these limitations. Noise2Fast uses a novel downsampling technique we refer to as ‘chequerboard downsampling’. This allows us to train on a discrete 4-image training set, while convergence can be monitored using the original noisy image. We show that Noise2Fast is faster than all similar methods with only a small drop in accuracy compared to the gold standard. We integrate Noise2Fast into real-time multi-modal imaging applications and demonstrate its broad applicability to diverse imaging and analysis pipelines.