In Galicia (NW Spain), granite was the most used stone material in historical buildings. Despite the good properties of granite as a construction material, it overcomes several physical and chemical weathering processes that cause decay, resulting in a loss of value in building materials, architectural elements and details. This is caused by a range of processes, from aesthetic damage to stone erosion. The causes of such decay are well known from case studies of historical buildings, being correlated with atmospheric agents, air pollution and aggregate materials, among others. In this work, we studied 15 historical monuments built with granite blocks of the architectural Heritage of the Barbanza Peninsula (Galicia). Because of the geographic features of this area, there is a steep rainfall and sea spray gradient that allows us to study historical buildings exposed to different environmental conditions in a short distance from the shoreline to inland. We used geochemical, petrological and microscopic tools to assess the decay of the granites and compared the results with environmental factors to assess the role of humidity (rainfall) and sea spray on the decay. Both the observation of coatings and the assessment of weathering have shown that buildings close to the shoreline are more affected by sea salts, while buildings far from the coast are mostly affected by biological weathering. Surprisingly, chemical weathering is higher in a strip area some hundreds of meters away from the sea shore and at lower altitudes (between 10 and 30 m). Indeed, very good correlation is observed for weathering indices, such as CIA, MWPI, VR, Si-Ti index, Kr, CAN and AKN, with a distance to the shoreline from 0.5 km, with linear correlation values ranging from −0.91 to 0.80.