Silver and gold are the most commonly used materials in optics and plasmonics. Silver has the lowest optical losses in the visible and near-infrared wavelength range, but it faces a serious problem—degradation over time. It has been repeatedly reported that the optical properties of silver thin films rapidly degrade when exposed to the atmosphere. This phenomenon was described by various mechanisms: rapid silver oxidation, sorption of sulfur or oxygen, formation of silver compounds with chlorine, sulfur, and oxygen. In this work, we systematically studied single-crystalline silver films from 25 to 70 nm thicknesses for almost two years. The surface morphology, crystalline structure and optical characteristics of the silver films were measured using spectroscopic ellipsometry, ultra-high-resolution scanning electron microscopy, and stylus profilometry under standard laboratory conditions. After 19 months, bulk structures appeared on the surface of thin films. These structures are associated with relaxation of internal stresses combined with dewetting. Single-crystalline silver films deposited using the single-crystalline continuous ultra-smooth, low-loss, low-cost (SCULL) technology with a thickness of 35–50 nm demonstrated the best stability in terms of degradation. We have shown that the number of defects (grain boundaries and joints of terraces) is one of the key factors that influence the degradation intensity of silver films.