Black crusts (BCs) are one of the most critical alteration forms found on stones belonging to architectural heritage. Since they could be considered as passive samplers of atmospheric pollution, it would be plausible to establish relations between the air contamination and the BCs. With this aim, we have characterized BCs collected on historic buildings from two Spanish cities (Granada and Vigo) with different polluted atmospheres, as well as formed on stone substrates of varied mineralogy and texture. Likewise, in order to assess the impact of the atmospheric pollutants on the growth of BCs, quartz fiber filters were used as surrogate substrates and placed nearby the studied buildings to collect and analyze the aerosol particulate matter (PM). To this end, an array of complementary analytical techniques was used to evaluate the mineralogy, chemical composition, and texture of the BCs and to establish the correlation with the ions, OC (organic carbon), and EC (elemental carbon) detected in the PM on the quartz fiber filters. As result, BCs developed on carbonate substrates from Granada show more complex structure than those from Vigo, which are thinner because of frequent rain episodes. In both cities, NaCl, Pb-Cl, and Ca-Cl-rich particles, Ca-phosphate particles and clusters of Ba-sulfate-rich particles were detected. However, metal-rich rounded particles were more abundant in Granada’s BCs, including soot particles. BCs from Granada were richer in carbonaceous components (OC and EC) than the Vigo’s BCs. Although in the filters PM did not show EC—mainly due to traffic—, in the BCs from both locations OC and EC were detected. Therefore, this different composition was related to the mineralogy of the stones and the higher pollution of Granada in contrast to the industrial and sea-exposed city of Vigo.