Two out of the four steps of risk assessment for chemical substances in food, i.e., exposure assessment and risk characterization, merit regional evaluation based on current legislation and local food consumption data. Therefore, mean and high exposures to food colors were estimated in Brazil using a conservative approach to screen substances with a higher risk of the exceedance of safety parameters. Brazilian National Consumption Surveys from the Household Budget Surveys (POF—Pesquisa de Orçamentos Familiares) from 2008–2009 and 2017–2018 were combined with the maximum permitted levels of 33 food colors. Higher exposure estimates were obtained for the oldest POF database. High priority for a refined exposure assessment was identified for six food colors for which the mean and high exposures were higher than the safety parameters, while medium priority was observed for eleven food colors for which the mean exposures were below but the high exposures were above the safety parameters. Low priority was noted for 16 substances for which no exceedance was obtained despite the conservativeness of the methodology applied. The prioritization of food colors for future risk assessments was achieved to identify substances for which more refined exposure methodologies are necessary to characterize the risk to health.