We use a long‐term record of ground‐based mid‐infrared (7.9–24.5 μm) observations, captured between 1984 and late 2019 from 3‐m and 8‐m class observatories (mainly NASA's Infrared Telescope Facility, ESO's Very Large Telescope, and the Subaru Telescope), to characterize the long‐term, multi‐decade variability of the thermal and aerosol structure in Jupiter's atmosphere. In this study, spectral cubes assembled from images in multiple filters are inverted to provide estimations of stratospheric and tropospheric temperatures and tropospheric aerosol opacity. We find evidence of non‐seasonal and quasi‐seasonal variations of the stratospheric temperatures at 10 mbar, with a permanent hemispherical asymmetry at mid‐latitudes, where the northern mid‐latitudes are overall warmer than southern mid‐latitudes. A correlation analysis between stratospheric and tropospheric temperature variations reveals a moderate anticorrelation between the 10‐mbar and 330‐mbar temperatures at the equator, revealing that upper‐tropospheric equatorial temperatures are coupled to Jupiter’s Equatorial Stratospheric Oscillation. The North and South Equatorial Belts show temporal variability in their aerosol opacity and tropospheric temperatures that are in approximate antiphase with one another, with moderate negative correlations in the North Equatorial Belt and South Equatorial Belt changes between conjugate latitudes at 10°–16°. This long‐term anticorrelation between belts separated by ∼15° is still not understood. Finally we characterize the lag between thermal and aerosol opacity changes at a number of latitudes, finding that aerosol variations tend to lag after thermal variations by around 6 months at multiple latitudes.