Background: Disability is an evolving concept that results from the complex interaction between a person with an impairment and the context in which he/she lives. There is limited understanding on the types, access and use of community assets valuable for people with disabilities, and the role of contextual factors in Colombia. Our goal with this work was to identify the factors at the levels of the socio-ecological framework, and their interaction, that influence the use of community assets among people with physical disabilities and community stakeholders in Envigado, Colombia. Methods: Using participatory mapping, a community based participatory approach, we carried out an assessment of community assets identified by people with disabilities and rehabilitation professionals. In-depth interviews (n = 32) informed the design of two participatory mapping activities, one among people with disabilities (n = 5) and a second with rehabilitation professionals (n = 4). Results were presented in a community forum to receive feedback on the findings. Results: Main findings indicate a chain of contextual factors that limit access and use of assets stemming from the personal (e.g. financial resources, inaccessible housing), interpersonal level (e.g. lack of a personal assistance or aid), and community levels (e.g. lack of accessible public transportation and inaccessible buildings). In most cases these barriers are heightened by system level barriers (e.g. lack of effective enforcement of the legal framework). Conclusions: Identifying these contextual factors, and their interactions, calls for stronger enforcement of the existing legal framework through articulated work between different stakeholders, so that people with disabilities can enjoy community assets.