For the creation of inclusive design solutions, designers require relevant knowledge about a diversity of users throughout the design process. Besides understanding users’ needs and expectations, the ways in which users perceive and experience the environment contain valuable knowledge for designers. Since users’ perceptions and experiences are mainly tacit by nature, they are much more difficult to communicate and therefore more difficult to externalize. Hence, more insight is needed into the ways designers can build knowledge on Universal Design through direct user contact. In a project called ‘Light up for all’ architecture students are asked to design a light switch and socket, elegant, usable and understandable to the greatest extent possible by everyone. Two workshops with user/experts are organized in the first stages of the design process in which students could gain insight into users’ experiences and perceptions through direct contact. Three data collection techniques are used to analyze the teams’ design processes: (1) a design diary, (2) observations of the workshops and (3) a focus group. By means of analyzing collected qualitative data, we have identified three different design aspects that affect designers’ UD knowledge building process. First, findings give indications on values and limitations of working with selected design artefacts when externalizing users’ experiences. Second, the value of stories clearly affected designers’ deeper understanding about users’ experiences. Finally, results show that in some situations, designers encountered contradictory information between observations and verbal conversations. These insights may help researchers to better understand designers’ process of building knowledge on UD from users’ experiences and perceptions, which may result in better incorporating users’ experiences when designing for everyone.