For several years now, women and minorities have got encouraged to choose physics programs and to enter physics faculties. Nevertheless, physics and in particular high energy physics, is a field of research strongly associated with cleverness and masculinity. Barely less elitist is the image of physics and physicists in general, be it in public (e.g. parents) and media (e.g. 'Big Bang Theory') or be it at school and university. Especially newcomers struggle with this imaginary gate keeper. Next to talent and diligence, physics students need a great deal of confidence and believe in their abilities as future physicists. It is therefore not enough to invite women and minorities to enter physics programs. Instead, we must understand the gendered, classed, and raced image of physics in its policy of knowledge-producing processes. And we need to overcome the reproduction of the 'hidden curriculum' while teaching physics. Therefore, we have to reflect on the power of norms and exclusions in the culture, representation, and teaching of physics. The aim should be to make diverse people feel more valued and welcome in physics.