Teachers represent a crucial dimension of social stratification in the school system. Their assessments of students matter by exerting short-term effects on students' motivation and achievement, but also by influencing students' educational prospects. Research shows that teachers' assessments are reasonably accurate. However, there is a residual component of inaccuracy emerging from individual -level factors (e.g., students' social or immigrant origin) as well as from classroom and school composition. In the present contribution, we add to this literature by providing a dual-process explanation on how contextual conditions on several levels, i.e., classroom, school, neighborhood and institution simultaneously frame teachers' expectations of their students. We test our theoretical model by using data comprising information on teachers, students, parents, and contextual settings of German elementary schools. We exploit institutional variations that influence the extent to which teachers perceive their expectations as relevant (i.e., binding versus non-binding recommendations for secondary school tracks). Results show that teachers' expectations are positively framed by a high share of students with immigrant background in the classroom. Yet, this effect is only valid for native students. Furthermore, we observe mutually reinforcing framing effects of the classroom's and the neighborhood's social composition on teachers' expectations. Finally, context influences are attenuated when teachers perceive their expectations as more relevant (i.e. track recommendations are binding).