The ability of microorganisms to degrade building materials depends on several factors. Biological corrosion occurs in close dependence with chemical and physical factors affecting microorganisms. The growth and development of microorganisms is stimulated by external stimuli, i.e., environmental factors. Microorganisms have a relatively large tolerance range for changes in environmental conditions. Under the right conditions, microorganisms thrive very well. The adverse effects may cause the inhibition of cell growth, damage, or lead to the death of the microorganism. Considering the impact of environmental factors on microorganisms, it is not possible to identify the most important of them. The result effect of overlapping factors determines the possibility of the growth of certain microorganisms. The main factors affecting the growth are temperature, humidity, hydrogen ion concentration in the environment, oxidoreductive potential, water activity in the environment, and hydrostatic pressure. This article provides a comprehensive overview of the factors causing biodeterioration. The influence of external/internal environment on the surface of cultural monuments made of mineral building materials, i.e., stone, concrete, mortar, etc., is presented.