Despite the effectiveness of screenings in reducing colorectal cancer (CRC) mortality, ~25% of US adults do not adhere to screening guidelines. Prior studies associate socioeconomic status (SES) with low screening adherence and suggest that neighborhood deprivation can influence CRC outcomes. We comprehensively investigated the effect of neighborhood SES circumstances (nSES), individual SES, and race/ethnicity on adherence to CRC screening in a multiethnic cross-sectional study. Participant surveys assessing 32 individual-level socioeconomic and healthcare access measures were administered from 2017 to 2018. Participant data were joined with nine nSES measures from the US Census at the census tract level. Univariate, LASSO, and multivariable mixed-effect logistic regression models were used for variable reduction and evaluation of associations. The total study population included 526 participants aged 50–85; 29% of participants were non-adherent. In the final multivariable model, age (p = 0.02) and Non-Hispanic Black race (p = 0.02) were associated with higher odds of adherence. Factors associated with lower adherence were home rental (vs. ownership) (p = 0.003), perception of low healthcare quality (p = 0.006), no routine checkup within two years (p = 0.002), perceived discrimination (p = 0.02), and nSES deprivation (p = 0.02). After comprehensive variable methods were applied, socioeconomic indicators at the neighborhood and individual level were found to contribute to low CRC screening adherence.