Power spectra of deprojected images of late-type galaxies in gas or dust emission are very useful diagnostics of the dynamics and stability of their interstellar medium. Previous studies have shown that the power spectra can be approximated as two power laws, a shallow one on large scales (larger than 500 pc) and a steeper one on small scales, with the break between the two corresponding to the line-of-sight thickness of the galaxy disk. The break separates the 3D behavior of the interstellar medium on small scales, controlled by star formation and feedback, from the 2D behavior on large scales, driven by density waves in the disk. The break between these two regimes depends on the thickness of the plane, which is determined by the natural self-gravitating scale of the interstellar medium. We present a thorough analysis of the power spectra of the dust and gas emission at several wavelengths in the nearby galaxy M 33. In particular, we use the recently obtained images at five wavelengths by PACS and SPIRE onboard Herschel. The wide dynamical range (2-3 dex in scale) of most images allows us to clearly determine the change in slopes from −1.5 to −4, with some variations with wavelength. The break scale increases with wavelength from 100 pc at 24 and 100 μm to 350 pc at 500 μm, suggesting that the cool dust lies in a thicker disk than the warm dust, perhaps because of star formation that is more confined to the plane. The slope on small scales tends to be steeper at longer wavelength, meaning that the warmer dust is more concentrated in clumps. Numerical simulations of an isolated late-type galaxy, rich in gas and with no bulge, such as M 33, are carried out to better interpret these observed results. Varying the star formation and feedback parameters, it is possible to obtain a range of power spectra, with two power-law slopes and breaks, that nicely bracket the data. The small-scale power-law does indeed reflect the 3D behavior of the gas layer, steepening strongly while the feedback smoothes the structures by increasing the gas turbulence. M 33 appears to correspond to a fiducial model with an SFR of ∼0.7 M /yr, with 10% supernovae energy coupled to the gas kinematics.