Flood damage estimation is a core task in flood risk assessments and requires reliable flood loss models. Identifying the driving factors of flood loss at residential buildings and gaining insight into their relations is important to improve our understanding of flood damage processes. For that purpose, we learn probabilistic graphical models, which capture and illustrate (in‐)dependencies between the considered variables. The models are learned based on postevent surveys with flood‐affected residents after six flood events, which occurred in Germany between 2002 and 2013. Besides the sustained building damage, the survey data contain information about flooding parameters, early warning and emergency measures, property‐level mitigation measures and preparedness, socioeconomic characteristics of the household, and building characteristics. The analysis considers the entire data set with a total of 4,468 cases as well as subsets of the data set partitioned into single flood events and flood types: river floods, levee breaches, surface water flooding, and groundwater floods, to reveal differences in the damaging processes. The learned networks suggest that the flood loss ratio of residential buildings is directly influenced by hydrological and hydraulic aspects as well as by building characteristics and property‐level mitigation measures. The study demonstrates also that for different flood events and process types the building damage is influenced by varying factors. This suggests that flood damage models need to be capable of reproducing these differences for spatial and temporal model transfers.