Health literacy is primarily understood as an individual construct. People with intellectual disabilities still seem to be a “hidden” population in health literacy research. A target-orientated health literacy approach for this population is needed for developing precise and effective interventions that consider social context dimensions. Therefore, the following research question was answered: Which dimensions influence the health literacy of people with intellectual disabilities? A secondary data analysis containing 38 guided interviews with people with mild to moderate intellectual disabilities was conducted. The analysis followed the content analysis by Schreier (2012). Six main dimensions were inductively outlined, which are “interpersonal relationship”, “organizations and communities”, “healthcare system”, “education”, “digital interaction spaces”, “politics”, and “cultural contexts”. All these dimensions influence people with intellectual disabilities on an individual level regarding their level of health literacy. The importance of these single dimensions becomes clear, although for now, the link between all these dimensions is not yet considered. In future research, the focus should be on how to develop interventions considering social context dimensions. Furthermore, analyzing the connection between those dimensions seems promising.