Individuals who are visually impaired encounter a number of challenges when attempting to orientate their own position or the position of others in relation to them within an unfamiliar indoor environment. To design an orientation assistive technology, it is crucial to understand the factors that reduce the user's sense of orientation. In this work, we discuss the disorientation factors that resulted from three user studies, which were conducted to formulate the basic requirements for an orientation assistive technology to assist visually impaired individuals in unfamiliar indoor areas. Using the feedback we elicited from one survey and two interview studies, we shed light on the factors that reduce the user's sense of orientation in unfamiliar buildings such as noise and traffic levels.