Abstract. Anomaly detection in imagery has widely been studied and enhanced towards the requirements of today’s available sensor data, whereas many of them require a background estimation in order to identify an anomaly or target. In this paper, we examine an analysis of simulation as background estimator for anomaly detection in thermal images of urban sceneries. We generate a surface temperature image and a sensor-like infrared image by combined image and elevation data and a thermal model suited for large scenes and fast simulation. With the simulated thermal image, we define anomalies as deviation between measurement and simulation. Pixel-wise image differencing of the measured and simulated temperatures and infrared images respectively are performed and evaluated concerning the full images as well as class-wise, including a material classification of the observed area. Our approach shows complementary results compared to RXD application on the measured infrared images. Metal roofs which appear warm in the thermal image and are not visually distinguishable from the residual image are detected.