There is a demand for ultra low-loss metal films with high-quality single crystals and perfect surface for nanophotonics and quantum information processing. Many researches are devoted to alternative materials, but silver is by far theoretically the most preferred low-loss material at optical and near-IR frequencies. Usually, epitaxial growth is used to deposit single-crystalline silver films, but they still suffer from unpredictable losses and well-known dewetting effect that strongly limits films quality. Here we report the two-step approach for e-beam evaporation of atomically smooth single-crystalline metal films. The proposed method is based on the thermodynamic control of film growth kinetics at atomic level, which allows depositing state-of-art metal films and overcoming the film-surface dewetting. Here we use it to deposit 35–100 nm thick single-crystalline silver films with the sub-100pm surface roughness and theoretically limited optical losses, considering an ideal material for ultrahigh-Q nanophotonic devices. Utilizing these films we experimentally estimate the contribution of grain boundaries, material purity, surface roughness and crystallinity to optical properties of metal films. We demonstrate our «SCULL» two-step approach for single-crystalline growth of silver, gold and aluminum films which open fundamentally new possibilities in nanophotonics, biotechnology and superconductive quantum technologies. We believe it could be readily adopted for the synthesis of other extremely low-loss single-crystalline metal films.