Thermoelectric conversion may take a significant share in future energy technologies. Oxide‐based thermoelectric composite ceramics attract attention for promising routes for control of electrical and thermal conductivity for enhanced thermoelectric performance. However, the variability of the composite properties responsible for the thermoelectric performance, despite nominally identical preparation routes, is significant, and this cannot be explained without detailed studies of thermal transport at the local scale. Scanning thermal microscopy (SThM) is a scanning probe microscopy method providing access to local thermal properties of materials down to length scales below 100 nm. To date, realistic quantitative SThM is shown mostly for topographically very smooth materials. Here, methods for SThM imaging of bulk ceramic samples with relatively rough surfaces are demonstrated. “Jumping mode” SThM (JM‐SThM), which serves to preserve the probe integrity while imaging rough surfaces, is developed and applied. Experiments with real thermoelectric ceramics show that the JM‐SThM can be used for meaningful quantitative imaging. Quantitative imaging is performed with the help of calibrated finite‐elements model of the SThM probe. The modeling reveals non‐negligible effects associated with the distributed nature of the resistive SThM probes used; corrections need to be made depending on probe‐sample contact thermal resistance and probe current frequency.