Selenium (Se) is an essential trace element for humans and its low or high concentration in vivo is associated with the high risk of many diseases. It is important to identify influential factors of Se status. The present study aimed to explore the association between several factors (Se intake, gender, age, race, education, body mass index (BMI), income, smoking and alcohol status) and blood Se concentration using the National Health and Nutrition Examination Survey 2017–2020 data. Demographic characteristics, physical examination, health interviews and diets were compared among quartiles of blood Se concentration using the Rao-Scott χ2 test. Se levels were compared between the different groups of factors studied, measuring the strength of their association. A total of 6205 participants were finally included. The normal reference ranges of blood Se concentration were 142.3 (2.5th percentile) and 240.8 μg/L (97.5th percentile), respectively. The mean values of dietary Se intake, total Se intake and blood Se concentration of the participants were 111.5 μg/day, 122.7 μg/day and 188.7 μg/L, respectively, indicating they were in the normal range. Total Se intake was the most important contributor of blood Se concentration. Gender, race, education status, income, BMI, smoking and alcohol status were associated with blood Se concentration.