Traditional light sources, e.g., incandescent and fluorescent lamps, are currently being replaced by white light-emitting diodes (wLEDs) because of their improved efficiency, prolonged lifetime, and environmental friendliness. Much effort has recently been spent to the development of Mn-doped fluoride phosphors that can enhance the color gamut in displays and improve the color rendering index, luminous efficacy of the radiation, and correlated color temperature of wLEDs used for lighting. Purity, stability, and degradation of fluoride phosphors are, however, rarely discussed. Nevertheless, the typical wet chemical synthesis routes (involving hydrogen fluoride (HF)) and the large variety of possible Mn valence states often lead to impurities that drastically influence the performance and stability of these phosphors. In this article, the origins and consequences of impurities formed during synthesis and aging of KSiF:Mn are revealed. Both crystalline impurities such as KHF and ionic impurities such as Mn are found to affect the phosphor performance. While Mn mainly influences the optical absorption behavior, KHF can affect both the optical performance and chemical stability of the phosphor. Moisture leads to decomposition of KHF, forming HF and amorphous hydrated potassium fluoride. As a consequence of hydrate formation, significant amounts of water can be absorbed in impure phosphor powders containing KHF, facilitating the hydrolysis of [MnF] complexes and affecting the optical absorption of the phosphors. Strategies are discussed to identify impurities and to achieve pure and stable phosphors with internal quantum efficiencies of more than 90%.