It is unclear whether the current average use of personal music players (PMPs) including mobile phones has affected hearing in the general population. The association between the use of PMPs and hearing loss was assessed in a large population cross-sectional and follow-up study with the following distribution: cross-sectional (2018): n = 26,606, 56% women, mean age 54 years and 20-year follow-up (baseline 1998): n = 12,115, 57% women, mean age at baseline 43 years. Hearing threshold was determined as pure-tone average over the frequencies 3, 4, and 6 kHz. We used linear regression to assess relationships between hearing threshold and PMP use (yes), duration (1–2/2–6/>6 h per week), or sound volume (low/medium/high), with nonuse as reference. The PMP use increased from 8% in 1998 to 30% in 2018. Compared with nonusers, neither use nor duration was related to hearing threshold. As to sound volume, listening at low levels was associated with better thresholds (−2.5 dB [−4.1 to −0.8]), while listening at high levels was associated with worse thresholds (1.4 dB [0.1 to 2.8]). We adjusted for age, sex, baseline hearing threshold, education, noise exposure, ear infections, head injury, and daily smoking. The association with sound volume was nearly twice as strong when adjusting for hearing threshold at baseline. Accordingly, the possibility of reverse causality was reduced although not eliminated by the follow-up design. This large population study showed no association between normal PMP use and 20-year progression in hearing; however users listening to high levels increased their hearing threshold.