Hearing loss (HL) is a global health problem with a high prevalence and profound socioeconomic impact. Pyrethroids are one of the most commonly used insecticides. Although previous studies have reported the relationship between pyrethroids and neurotoxicity, little is known about the effect of pyrethroid exposure on the auditory system among the general population. This study is aimed to investigate the association of pyrethroid exposure with hearing threshold shifts of adults in the United States. A total of 726 adults, aged from 20 to 69 years from the 2011–2012 National Health and Nutrition Examination Survey (NHANES) data were included in the study. Urinary 3-phenoxybenzoic acid (3-PBA), a general pyrethroid metabolite, was used as a biomarker for pyrethroid exposure. HL was defined as a pure-tone average (PTA) at 0.5, 1, 2, 4 kHz ≥ 20 dB in the better ear. Analyses by using multivariate linear regressions were conducted to explore the associations of urinary 3-PBA with PTA hearing threshold shifts. There were no statistically significant correlations between Ln-transformed 3-PBA and either low-frequency or high-frequency hearing thresholds after adjusting for age, gender, race/ethnicity, education level, firearm noise exposure, occupational noise exposure, recreational noise exposure, serum cotinine, BMI, hypertension, and diabetes. However, associations of 3-PBA with both low-frequency and high-frequency hearing thresholds depended on age (P interaction < 0.0396 and 0.0017, respectively). Positive associations between Ln-transformed 3-PBA and both low-frequency and high-frequency hearing thresholds were observed in participants aged 20–39 years after adjusting confounders (β = 1.53, 95% CI: 0.04–3.01, and β = 3.14, 95% CI: 0.99–5.29, respectively) with the highest tertile (≥ 0.884 μg/g creatinine) of 3-PBA compared with the lowest tertile (< 0.407 μg/g creatinine). The possibility of interaction between 3-PBA and age on the hearing threshold shifts indicated that pyrethroid insecticides were prone to be more toxic to auditory system in younger adults than in older ones. Further studies will be required to confirm these findings.